Automated Image Recognition for inserting Alt Tags

[dvsDave]

Looking to automate adding alt tags to uploaded images, and retroactively apply automated alt tags to existing uploads.

Amazon has a service called Rekognition that looks like it could accomplish this. https://aws.amazon.com/rekognition/
Google also has a service called their Vision API: https://cloud.google.com/vision/docs/labels

Ideally, the results could be presented to the user for verification and editing. A box that says, here's what we came up with for image tags, do you agree or want to make changes?

Requirements:

1. Automatically trigger on image upload
2. Confirm with the poster that the image tags are relevant, provide editing mechanism if not.
3. Retroactively scan and update existing images
4. Provide Admin Panel interface for API credentials and retroactive scanning (with optional limits for # of images processed per month)
5. Support for 2 Years, keeping the app compliant with current XF version (2.1 and up), unless future XF built-in functionality renders all aspects redundant.
6. The developer is free to sell the app in the Resource area, but initial backers/investors get a lifetime license.

PROGRAMMERS:

If you have more questions, please reply to this thread.
If you want to submit a bid, please send me a PM with the following:

• Bid Price
• Portfolio of Work (must not be first project)
• Milestone breakdown in parts (a portion of the award will be split at the demonstration of each milestone)
  • Successfully triggering upon upload and receiving back tag(s)
  • Prompting user to accept or edit tags
  • Building a Control Panel interface for inserting AWS and/or Google Vision credentials
  • Building a system for retroactively scanning for older images in the server, setting max limits of images processed per month
  • Demonstrate proper install on a fresh XF install

 

[Ozzy47]

I recommend @TickTackk if he is interested in the job. Quality code and reasonable prices.

 

[dvsDave]

I recommend @TickTackk if he is interested in the job. Quality code and reasonable prices.

I've talked to @TickTackk and he's swamped till January. There have been a couple of developers who have contacted me, still getting quotes, nowhere near ready to award a contract to someone yet.

 

[Ozzy47]

Too bad, I think he would have been a good fit for this.

 

[Forsaken]

I've talked to @TickTackk and he's swamped till January. There have been a couple of developers who have contacted me, still getting quotes, nowhere near ready to award a contract to someone yet.

TickTackk would probably be one of the better devs to do this due to how focused he is on performance, and quality. Hopefully you can find another fit.

 

[djbaxter]

I had forgotten but I contacted @TickTackk back in September about the same thing, a custom addon to insert alt tags for images, and after looking into it this was his reply:

Wait. You don't even need this add-on. XF 2.2 already provides the ability to set alt of an image.

First upload an image then click on "Insert..." then on "Full image"

1. Upload an image
2. Click on "Insert..."
3. Click on "Full image"
4. Click on the inserted image
5. Click on
   
   
   
   icon
6. And you should be able to set the alt title

I then asked:

Okay but is there a way to do this automatically from the attachment title?

His reply:

Not sure what you mean by "attachment title" but if no alt title value is provided then the filename is used.

Me:

But unless I'm missing something, that requires an additional step to add an alt title tag to the posted image, which most forum members probably wouldn't do. Hence the request for automatically adding a title tag from available information when an image is uploaded.

Oh. It does insert the filename as an alt tag for an attachment: [ATTACH type="full" alt="squirrel-dress.jpg"]235949[/ATTACH]
But not if uploaded as an attachment instead of an image: [ATTACH type="full"]235950[/ATTACH]

TickTackk:

That is expected but if you check the output it generates in HTML, it will have the alt tag value set to the attachment file name.

So it's not entirely transparent but you can see it in action if you check the source code in XF 2.2.x.

 

[btmgreg]

This is very well thought out. Something I'd love to see in core or sold (or free haha)

 

[Forsaken]

This is very well thought out. Something I'd love to see in core or sold (or free haha)

There's a lot of usability improvements that the core needs... I manage and maintain a private forum for blind users for my stepdad that will have to be stuck on XF1 because of the amount of changes I did to it, and custom add-ons that were made for it.

Ironically, XF is still better than IPB, vB and WBB. Discourse/Flarum are somewhat usable from the topic list view as you're not having to navigate through a large hierarchy structure, but their node/forum view can be difficult. More over the scroll to load for new posts in discussions is very bad for screen readers normally.

There's a reason most disabled people (especially blind/partially sighted) people still keep to newsgroups and similar services that are content focused.

 

[btmgreg]

Would you care to share the link to the private forum? or a screenshot? I'd be very interested to see what usability changes and custom addons are used to make peoples life easier. I imagine it's a whole different world for what I and many others are used to. I also want to make sure that I can cater to my audience on my site who have similar problems - I really do wish you well in getting this off the ground and working. Well done.

 

[Forsaken]

Would you care to share the link to the private forum? or a screenshot? I'd be very interested to see what usability changes and custom addons are used to make peoples life easier. I imagine it's a whole different world for what I and many others are used to. I also want to make sure that I can cater to my audience on my site who have similar problems - I really do wish you well in getting this off the ground and working. Well done.

I'll ask my stepdad, but it's not something that anyone would use for their own community unless it's primarily for someone who is blind/partially sighted. There is only one category hierarchy, the focus is on forum name/description but no last post and additional data.

The whole point is to be data-forward, rather than to look pretty or to have anything that is additional (like sidebars). It's a one column layout, and if the information is not critical to the content it has been completely removed.

As far as custom add-ons go, the most important one is one that was to make it easier to navigate purely with mouse/keyboard, with additional functionality for quick copy and pasting of post content, or post permalink. I also had to rewrite all the font-size to be REM-based as XF1.5 was a nightmare for font-size scaling.

The community is mostly for tech support for phones/computers for the blind, and other than myself there are only ~10-15 people who can see (we just use the default style).