Lack of interest DKIM: Change location of key material

[Kirby]

Currently, the private key is stored unarmored in internal-data://keys, I am not sure if I'd be too happy to have a private key stored in an (unstrusted) public cloud object storage.

Furthermore, the key could be accessed pretty frequently (on busy forums) - in which case it would always be downloaded.
I'd therefore like to suggest to store this in another place (code-cache://, data registry, etc.)

 

[digitalpoint]

How are you currently handling directories inside internal_data? In my setup, I'm making temp and code_cache stay local, but everything else uses Gluster. I would think you aren't globally putting everything in internal_data into the cloud (seems like a terrible use of resources for something like the temp directory). So whatever you are doing to handle that, couldn't you do the same for the keys directory?

 

[Kirby]

In my setup, I'm making temp and code_cache stay local, but everything else uses Gluster.

We use the same setup.

I would think you aren't globally putting everything in internal_data into the cloud (seems like a terrible use of resources for something like the temp directory).

I wouldn't do that either, but I think there are customers that do run internal-data on S3.

So whatever you are doing to handle that, couldn't you do the same for the keys directory?

While it is possible to configure code-cache separately (by default it is internal_data/code-cache), this is not possible for keys.
Persoanlly I don't care too much (we don't really need this feature anyway as we're signing via Postfix & opendkim-milter), just bringing this to attention for others customers that do use cloud storage

 

[digitalpoint]

Ya, kind of in the same setup... in my original suggestion, I was just thinking about storing the key within an option (probably a hidden one for the key itself). And I also don't care too much because I'm doing it all at MTA level as well. I suppose the internal_data file system is probably the better place for the key purely from a security standpoint, but it does make for some configuration annoyances for people with non-default internal_data setup. But I guess if you are running a non-standard internal_data setup, you probably are already handling similar things (like the temp directory there).

 

[Kirby]

I suppose the internal_data file system is probably the better place for the key purely from a security standpoint, but it does make for some configuration annoyances for people with non-default internal_data setup.

Hmm, maybe I am missing smth. but as far as I can see it currently "not possible" to configure internal-data://keys to be stored on another filesystem than internal-data://.
So if that is S3, the key file would be stored there as well.

How would you configure internal-data:// to be on S3 but not internal-data://keys?

 

[digitalpoint]

Would require some trickery with an addon that is overriding the filesystem adapters most likely. I don't think there is a way to override just the temp directory, so it's part of the internal_data filesystem adapter. I just bring that up because if someone is using a cloud storage system for internal_data, I would hope they have managed to figure out a way to not be reading/writing temp stuff (for things like image resizing/cropping). However they are doing that (which will be unique to their setup), they could just do that for keys as well.

 

[briansol]

Idiots like me ARE using s3 for all internal_data because we don't have a way to do it on a subfolder-by-folder basis as noted. The extra hit is noted, but also not too large of a bump on my site. Maybe it costs me 10 cents more and slows down the experience 1% of the time (uploading an image).

So, i support this idea and also maybe the real feature request is to change s3 adapter or even the internal data path config to have more options (not root level, but sub folder level) via config.php.

 

[digitalpoint]

You know what would be nice and solve not only this problem but some other problems? A container that mapped sub-directories of internal_data to file system adapters (could just be all mapped to local by default). That would allow sites to override the file system adapter to be used in a more granular way than the current form of just "everything in internal_data".

$container['internalDataFsAdapters'] = function (string $c)
{
      return 'XF\LocalFsAdapter';
};

Then someone could set various file system adapters for the sub-directories individually by changing their config file (or you could do it automatically as part of an addon)... something along the lines of:

$c['internalDataFsAdapters'] = function(string $c)
{
   $override = [
      'addon_batch' => 'Gluster\PoolAdapter',
      'attachments' => 'AWS\CloudStorageAdapter',
      'file_check' => 'Gluster\PoolAdapter',
      'image_cache' => 'Gluster\PoolAdapter',
      'imported_xml' => 'Gluster\PoolAdapter',
      'keys' => 'Gluster\PoolAdapter',
      'oembed_cache' => 'Gluster\PoolAdapter',
      'sitemaps' => 'Gluster\PoolAdapter',
   ];
   return (empty($override[$c]) ? 'XF\LocalFsAdapter' : $override[$c]);
};[

Like maybe I want to have most things stored on the local network via Gluster, but attachments stored in the cloud and then code_cache and temp stored on the local machine.

This would solve the underlying issue you have with the keys directory, and also solve some other issues (and it's scalable because you can override future directories that don't exist yet. Also solves the issue where people like @briansol want to use S3 for attachments, but maybe don't want to be reading/writing the temp directory to/from the cloud.

 

[Kirby]

I think there might be some confusion how XenForo storage does work:

There is a subtle difference between internal_data local directory and internal-data VFS (which by default uses a local adapter on internal_data).

Temporary files will always be written to and read from internal_data/temp, even if internal-data is configured to use another adapter (S3, etc.) - unless a different directory for temporary files is explicitly configured via

$config['tempDataPath']

Likewise, cached code is always written to code-cache (which by default uses a local adapter on internal_data/code_cache) - unless a different adapter is configured for code-cache and read access to the local directory is configured via

$config['codeCachePath']

.

So it doesn't really need trickery to keep those two local while having everything on internal-data on S3, in fact this is standard behaviour.

Anyway, my suggestion was just to move the key from internal-data to somewhere else as I really think the current implementarion isn't that great.

Being able to granularly use different adapters for different parts/paths within existing VFS is a nice idea too, but I think thhat should be an entirely separate suggestion?