Not all apples are equal. In the industrial farming context, apple-trees are usually harvested only once a year, while apple maturation can happen over several one to two weeks. For economic reasons harvesting the tree several times is not advantageous for the company. However when correctly sorted under-ripe or over-ripe apples can be used too (for juices, fruit purée, transformed products etc). Instead of throwing away perfectly usable apples this can give them a second chance. So it all boils down to correctly sorting your fruits but the problems is usually we are talking a lot, lot of fruits…
In order to improve the process we can imagine a machine based on image recognition (and artificial intelligence) in order to sort the fruits!
The idea is to consider each apple individually and to take several picture of the fruit. Than an image recognition app will indicate the presence of possible defects and the predominant color (and thus ripeness) of a certain part of the fruit. A meta-algorithm combining all this information will than take a decision on whether the apple is suitable to be sold as it is or it should be transformed.
We will use the ml5.js which is user-friendly library providing access to machine learning algorithms. With deep learning we can teach an artificial intelligence to recognize different elements in a picture and event spot characteristics like a little sport on the skin on an apple. In order to do so we must provide a large data base of already classified images. In our case it would be useful to have a date base whit different tones an apple can take (greens, pinks, reds, oranges, browns etc) for instance. Once the IA has analyzed the database it’s time to test it: providing a non-classified image and checking the result.
Off course no algorithm is perfect…
That why it is important to improve your AI by “feeding” it with as many examples as possible!
In order to use the ml5.js library. You can learn more about it here.You need to either download it (which will cost you space but will allow you to use the program offline) or to simply source it the HTML part of you document. In the following example we chose the second option:
We than code the part in JavaScript where we work with the ml5 library. We have essentially two variables we are dealing with the image we want to classify and the prediction made. We than made a function to retrieve the result. Here is a possible piece of code:
In order to go more in depth on how to precisely code this we recommend this video :
It will be off course very accurate to take more than one photo of the apple in order to decide what to do with the apple (a ripe apple can have a green spot somewhere on its surface for instance).
Here is an example of an (more sophisticated) already existing machine of this sort: