Eval arbitrary code
You can allow running arbitrary code at hook locations, by evaling the code (or including a file) This is one of the most flexible solutions for plugin developers, since it allows you to do pretty much anything. Disadvantages: Uses eval and potentially pollutes the scope where the hook is.
Use "filter" functions
Another approach assumes that plugin developers usually need to modify the contents of certain variables. You create a plugin as a function, and that function is registered for the hook location you've chosen. Calling the hook then consists of calling all functions registered for that hook, while passing them references to the relevant variables for that hook (which variables are available for which hooks would have to be documented, or look in the code). In your function, you do stuff to the variable(s) and you're done. Or if the main code has an if branch based on a decision taken in the hook code, then the function can return a value. For each hook location, the system would pass pre-defined variables to the hook functions. Depending on what the code is doing at the hook location, the relevant variable(s) would be passed. Benefits: No eval, no polluting the global scope. Disadvantage: not quite as flexible as option #1, since you're not in the same scope as the parent code. But there are workarounds to that-- globalising variables, or pulling in instances of singleton classes. The difficulty would be accessing local variables in the parent scope, if they were not passed to the filter function. [Notes: see the MediaWiki hook system]
Use an API of sorts
Expose as many actions as possible via an API, and products/plugins would accomplish things via the API instead of directly modifying the internal code with their plugin. This is less flexible, but offers great resiliency to refactoring. The internal XF code could change completely, and as long as the API interface stays the same, the plugins would continue to work. When I say API here, I'm referring to a set of classes/functions that client code could call to accomplish things. Could be called an SDK too, I guess. A remote API (via HTTP) is a separate animal (which would also be nice).
Should there be the option (or requirement) for file-based plugins/products so that the code can be included instead of evaled? Should products be designed in such a way that a product is a class and each plugin/hook code is a function in that class (to be called by a filter/hook location)?
Bringing it together
I'd say have an API for all the common stuff people want to do. Plugins/products would use the API for everything that they can. If they want to do things that aren't possible with the API, then have the "filter" functions throughout the code base in strategic locations. If a need is shown for injecting arbitrary code anywhere in the current scope, then consider the first option.