Revise all the developer docs (and maybe even the normal docs) using AI

[RobinHood]

So I've been creating some docs recently for stuff I've been building, and AI is a freakin' boss when it comes to analysing code bases and writing really clear and comprehensive docs.

I just posted a small repo for an internal tool and it was able to produce some super high quality docs really fast for quick start, usage, architecture, troubleshooting. Combined with some steering to add reasoning about why I had done certain things, the docs turned out so much better than if I was to try and write something from scratch.

It even caught a few things that it put into the docs, where I realised it wasn't quite right or could be improved in the codebase for usability for a new user, so I updated both the codebase and the docs. As an iterative process, its was great.

Coincidentally Anthropic released a neat video demonstrating a much more hardcore version of this!

To view this content we will need your consent to set third party cookies.
For more detailed information, see our cookies page.

Accept third party cookies

 

[QuackieMackie]

So I've been creating some docs recently for stuff I've been building, and AI is a freakin' boss when it comes to analysing code bases and writing really clear and comprehensive docs.

I just posted a small repo for an internal tool and it was able to produce some super high quality docs really fast for quick start, usage, architecture, troubleshooting. Combined with some steering to add reasoning about why I had done certain things, the docs turned out so much better than if I was to try and write something from scratch.

It even caught a few things that it put into the docs, where I realised it wasn't quite right or could be improved in the codebase for usability for a new user, so I updated both the codebase and the docs. As an iterative process, its was great.

Coincidentally Anthropic released a neat video demonstrating a much more hardcore version of this!

To view this content we will need your consent to set third party cookies.
For more detailed information, see our cookies page.

Accept third party cookies

I completely agree that the developer docs need a rehaul. A lot of the content is over two years out of date, and even the copyright footer still says Developer documentation for XenForo® © 2010–2023 XenForo Ltd. It’s pretty clear the docs haven’t been getting much love.

I also agree that using AI for documentation is extremely helpful, up to a point.
I use Junie all the time to search XenForo's codebase for specific features. It’s great for finding bits I want to make changes to, but you definitely still need to check everything. If AI generates templates for example, double-check that the html tags actually exist. It rarely gets everything correct, but for writing the explanatory text, it’s excellent with a bit of tweaking.

Honestly though, I’d suggest being the change you want to see. The dev docs are open source (sort of), and XenForo encourages community contributions: https://github.com/xenforo-ltd/docs/pulls.

 

[RobinHood]

Yeah, that's why I mentioned it needs to be iterative you and need to carefully curate it as the operator of the agent. But there's no doubt that someone could easily spend a couple of days bashing out a complete overhaul in a fraction of the time it would take to do manually using modern tools like this.

It needs to be someone with need intrinsic knowledge of what's really in the code base and what they know would be of use to onboard new developers to the platform to help guide and mould the output.

 

[RobinHood]

It could also then integrate an llms.txt to help developers ingest the docs into their ai coding workflow to ensure best practices are followed for add on development

 

[QuackieMackie]

Yeah, that's why I mentioned it needs to be iterative you and need to carefully curate it as the operator of the agent. But there's no doubt that someone could easily spend a couple of days bashing out a complete overhaul in a fraction of the time it would take to do manually using modern tools like this. Especially if it's someone with need intrinsic knowledge of what's really in them to help guide and mould the output.

A dev doing this would be ideal, but with 2.4 in the works, I imagine they’d prefer to wait, otherwise, they’d have to redo it. Personally, I think updating the dev docs should have been a mandatory part of each update.

I don’t mind updating the sections I contribute to as I discover features, but that approach feels a bit like the blind leading the blind.

It could also then integrate an llms.txt to help developer ingest the docs into their ai coding workflow to ensure best practices are followed for add on development

As for adding something like llms.txt, it would be nice if AI was up to date if I asked it a question. But i'd prefer to get actual up to date content before worrying about that.

 

[RobinHood]

Although considering you dont' get access to the code until you buy it, there may be a case to be made for having an llms.txt for the core codebase vs just the docs.

 

[QuackieMackie]

Although considering you dont' get access to the code until you buy it, there may be a case to be made for having an llms.txt for the core codebase vs just the docs.

I don't see why adding a llm.txt to the dev docs would be a problem. The dev docs contain none of Xenforos actual source code (to my knowledge, and if it does I imagine it's very minor.) Giving AI access to the dev docs would let AI possibly gather the information it needs to generate semi okay templates, and structure controllers right.

 

[RobinHood]

I think updating the dev docs should have been a mandatory part of each update.

I agree, and it should be relatively simple with the aid of an llm once a compressive core docs is deployed.

IMO it's a pretty easy win to help onboard more external developer to help scale the amount of code contributed to the XF ecosystem, which would make a lot of sense to make it a priority considering the slow pace of the core development for the last while.

Getting more 3rd party devs launching exciting add ons is great for the ecosystem and takes the pressure off the core team for a lot of stuff if those niche request can be filled more easily.