Even if its nerfed and abandoned, a heartening new trend that I’ve seen is cutting edge technology being democratized in FOSS apps. There’s all sorts of ai stuff and image stuff that was limited to the top five tech companies a year ago that’s now available to everyone.

isn’t it just a new way to access screen based search? i mean, i can do things you tak about by launching google assistant (non-gemini) and choosing “search on my screen”, i use it daily to translate difficult japanese in my dulingo, select text and search on my non-pixel/samsung phone, it’ there since few years AFAIK, i don’t think it’s going anywhere, because reverse image search is the future, also you’re training their neural networks that way(👉ﾟヮﾟ)👉

The ‘translate what’s on my screen’ was a thing that google assistant could do around Android 8. I found it very handy when living abroad and not really knowing the language.

That one didn’t use OCR, but could only do text in UI elements. Don’t know why they killed it. Maybe some security reasons caused the API that reads another apps UI elements to be canned.

Do you know if there’s a FOSS project that does any of this? Android does provide a screen capture API, and there are some FOSS OCR and translation apps separately, bit I don’t know of one that combines these things.

I think it has gotten better since it was first released for Samsung. The new translate button is very good.