Image recognition on localhost not google/apple cloud based [solved]

graysky · 2017-12-17 17:57:01

I've googled a bit and found that digikam provides facial recognition on the local host, which I am experimenting with. I'm wondering about more powerful indexing akin to what apple does where objects/places etc. (2,000+ different ones as I understand it) are recognized in photos and are available as search terms. Thanks for any suggestions.

Last edited by graysky (2017-12-18 19:24:53)

Trilby · 2017-12-18 17:16:01

Do you want a polished user-friendly "app", or are you just looking for a pre-trained object-classifier?

graysky · 2017-12-18 18:20:59

@Triilby - Just something workable. Can have a CLI if needed.

B1omman · 2017-12-18 18:31:06

I do not know of any such readily available application for end users
and probably there are none. However, what you are asking for can
technically be achieved to some extent, but not in any easy way.

For location, you would have to find some tool to read metadata from
images, i.e. exif-data and geotags etc.

Recognition of various degree is where you will have to do some hard
work probably. Pre-trained models on github might point you in the
direction where you will have to look.

What Apple and Google have that makes this possible for them is
data, lots of data. Basic feature extraction from images will not be
that hard though, such as finding faces or finding signs. You might
even be able to read the signs if you build a model good enough,
or possibly find some that suits your input.

Trilby · 2017-12-18 19:02:15

I just looked a little bit at the apple page - I've never used any of that, but it's not clear to me whether their search is actually doing any object recognition/classification on the images themselves. From the description it could just be using the date and location of the photo taken with the phone's camera along with all the other data in the phone about your location, travels, calendar, etc.

It sounds a bit more like the Google Now Cards than a image processing tool.

graysky · 2017-12-18 19:07:42

@Triilby - No, it's actually pretty amazing. The wife has an iphone and the image search can discriminate between dog breeds. I read somewhere that it categorizes thousands of objects.

EDIT: https://medium.com/@iosight/behind-appl … f3f617d31c

If I paste that list into a text file, it's over 4,400 (although there are quite a few singular/plural terms):

sed 's/, /\n/g' list | wc -l
4431

Last edited by graysky (2017-12-18 19:53:57)

Trilby · 2017-12-18 19:15:51

Ah, in that case it is an image classifier. There are some freely available ones out there, but they will not be nearly as good as apple's or google's.

Training a machine learning model to do this well is computationally intensive: for a frame of reference, perhaps comparable to compiling a kernel. You certainly can do it on your average home computer or laptop now, but you will not be doing it on the fly to label images. But once the model is trained, it can be used to categorize a new image virtually instantaneously. So apple just has a well trained model/classifier.

You can install tensorflow and set this up yourself without much trouble. Certainly finding a good model is not trivial, but the oddity of many such machine learning models is that having a good model isn't nearly as important as having ridiculous ammounts of training data. And that is what you'd never really be able to get (and that is what google collects from you every time you do one of those reCAPTCHA things).

And there's the issue with the pretrained models you could find out there. They'll do well enough to be a pretty cool gimmick - but they won't do nearly well enough for your wife to not miss her apple version, or even to take your version very seriously.

It's easy to find good free and open source software (TensorFlow is top-of-the-line software to implement this sort of thing, and it's in our repos!) Finding large quantities of training data, on the other hand, is never free: this is the sort of data the big tech companies hord and/or invest huge ammounts of money to collect.

Last edited by Trilby (2017-12-18 19:18:36)

graysky · 2017-12-18 19:24:35

Thanks for the replies, all. This is probably one of those times to just use google AI.

Morn · 2017-12-18 19:49:00

Trilby wrote:

It's easy to find good free and open source software (TensorFlow is top-of-the-line software to implement this sort of thing, and it's in our repos!) Finding large quantities of training data, on the other hand, is never free: this is the sort of data the big tech companies hord and/or invest huge ammounts of money to collect.

It looks like the STL-10 and CIFAR-10/-100 datasets are freely available and a good place to start. If you take a lot of pictures of dolphins and lobsters they might even be useful, but probably less so for distinguishing different dog breeds.

Somebody should convert all Wikimedia Commons photos into one giant labeled machine learning dataset, train a Tensorflow model on it and release the trained network as a plugin for Linux photo editing software. The Open Source dog breed classifier needs to be a thing, Internet!

ppenguin · 2018-03-20 17:52:35

Sorry for sort of necro-bumping, but you all seem to be knowledgable.
I was just wondering: I have started using digikam for categorising faces, but find the performance of recognising _known_persons_ after training still not so good.
I can imagine that the backend was developed before tensor flow, so it seems like a good idea to integrate tensor flow in the face recognition code to improve performance at relatively limited effort?
Would that be feasible?

Cheers,

ppenguin

Trilby · 2018-03-20 19:00:00

Again, the limiting reagent is not the quality of the algorithm. Tensor flow is only marginally "better" than it's predecessors, it's just much more efficient.

But in either case, the limiting reagent is training data. Despite Morn's suggestion above, I don't think using all the images in Wikimedia Commons would put much of a dent in the training data needed to get a well trained model.

So, does your collection of face images for training data number in the millions or in the billions? If the answer is neither, then I'd really not have high expectations for the result.

ppenguin · 2018-03-20 19:25:35

Thanks for the prompt reply.
Wow, I really didn't expect that such a large data set would be necessary. But, come to think of it, if the case is that there is a limited set of faces (user's image collection), within which one trains the system to put connect certain names to selected faces, the data pool is actually the same for google/apple/local? (I mean just considering face _recognition_, not face _detection_)

On a side note, I thought picasa used to do the job reasonably well (better than digikam anyway), I guess google might have used some "early AI" algorithms with picasa?

Trilby · 2018-03-20 19:34:30

No, the google/apple products are trained on vast arrays of data in order to first identify faces, then to identify the types of features in images that differentiate individual human faces. Only then it is tasked at the much simpler task of sorting a bunch of images of the few individual human faces in your image library.

You could absolutely train a model to distinguish between a small number of faces within your current image library, but the problem is it would either fail to categorize them well, or it would overfit the model to the existing training samples and thus do quite poorly on any new image.

ppenguin · 2018-03-20 20:03:20

Thanks, that's helpful indeed.
In other words, it also wouldn't make much sense to separate detection and recognition because my recognising intelligence needs to be based on vast "experience" anyway.
But luckily if I google for "face" in google image search, I get a lot of potential random training data to do that (although I'd probably need varied queries to find faces which are more blended into their environment.
Then I'd "only" need to manually teach the AI the "known" faces within the aggregate collection of random and own-collection faces, while (cumulatively) filtering out the unknown faces. Probably I need to double back then to fish out some false negatives from the rejected pool, but that could be a proposition.

Hm, I was looking anyway to check out tensor flow, so I might try my hand to hack it into digikam anyway. (My main problem is lack of time unfortunately).
As I mentioned, what struck me most is the much better face recognition of picasa vs. digikam with a similar dataset, there must be a reason for that.

Thanks for the insights!

Trilby · 2018-03-20 21:36:45

There are many instances of comparin apples to oranges here to the point I'm not sure how to, or what to respond too.

You say your goal is to differentiate faces of different people in your images. But you keep mentioning how much better picasa is than digikam at differentiating faces from non-faces. These are different problems.

ppenguin · 2018-04-15 15:55:14

Sorry, completely missed your (prompt) answer.
I might have expressed myself poorly. I meant differentiating _faces_ of different people (not non-faces). What I want is better _recognition_ in digikam, i.e. correct allocation of the known name (which I told the program while manually training it) to the not-yet tagged faces.
Face _identification_ (recognising a face for a face) is good enough for me with digikam.

I often get proposals with the wrong person name, even when the picture is almost a duplicate of one I already assigned the correct name to. (To be fair, there are pictures with other persons that have the same angles and lighting, and also familiar connection, so this is a complicating factor), but it was this performance that I found so good in picasa.

Last edited by ppenguin (2018-04-15 16:05:04)

Arch Linux

#1 2017-12-17 17:57:01

Image recognition on localhost not google/apple cloud based [solved]

#2 2017-12-18 17:16:01

Re: Image recognition on localhost not google/apple cloud based [solved]

#3 2017-12-18 18:20:59

Re: Image recognition on localhost not google/apple cloud based [solved]

#4 2017-12-18 18:31:06

Re: Image recognition on localhost not google/apple cloud based [solved]

#5 2017-12-18 19:02:15

Re: Image recognition on localhost not google/apple cloud based [solved]

#6 2017-12-18 19:07:42

Re: Image recognition on localhost not google/apple cloud based [solved]

#7 2017-12-18 19:15:51

Re: Image recognition on localhost not google/apple cloud based [solved]

#8 2017-12-18 19:24:35

Re: Image recognition on localhost not google/apple cloud based [solved]

#9 2017-12-18 19:49:00

Re: Image recognition on localhost not google/apple cloud based [solved]

#10 2018-03-20 17:52:35

Re: Image recognition on localhost not google/apple cloud based [solved]

#11 2018-03-20 19:00:00

Re: Image recognition on localhost not google/apple cloud based [solved]

#12 2018-03-20 19:25:35

Re: Image recognition on localhost not google/apple cloud based [solved]

#13 2018-03-20 19:34:30

Re: Image recognition on localhost not google/apple cloud based [solved]

#14 2018-03-20 20:03:20

Re: Image recognition on localhost not google/apple cloud based [solved]

#15 2018-03-20 21:36:45

Re: Image recognition on localhost not google/apple cloud based [solved]

#16 2018-04-15 15:55:14

Re: Image recognition on localhost not google/apple cloud based [solved]

Board footer