Nov 30, 2021
“NutrInvest”
This project was carried out as part of the Techlabs “Digital Shaper Programs” in Dortmund (summer term 2021)
In a nutshell:
NutrInvest is an Artificial Intelligence (AI) application. With this application, the user can upload a food picture (e.g., apple, bread, beef, etc.). Once uploaded, the image is sent to an AI database using an image classifier. Once the AI identifies the food in the picture, it extracts the nutritional value data. The latter is sent back for the users to view in the interface of the application.
Introduction:
Our journey started with the submission of ideas that could be concretized within the framework of a web development project which would feature works combining multiple skills and talents from the 2021 digital shapers cohort of Techlabs Dortmund. The participants selected the project they were interested in. The members of the team developing this application were convinced right from the start of the usefulness of this application and the impacts it could have in the field of health and well-being.
NutrInvest, as it indicates, highlights the value we accord to and how we invest in a healthy lifestyle. People who are doing exercises, those who are on a diet, and even those who just want to balance the nutrition in the food they are eating need practical solutions that would help them make a quick decision in terms of what to eat and how to eat healthily.
In the beginning, we were imagining a simple solution that would allow users to have easy access to the nutritional data of a specific food without typing the names of the food and doing lengthy research online. Easy access is crucial, and they can do this when at the supermarket, in public transportation, at the restaurant, even in rural areas where there is the internet, etc.
As a solution, we wanted to combine our skills in AI, frontend, and backend, developing an application responding to those needs. With a picture and well-trained data, AI can be a powerful tool that would allow us to get quickly a highly precise nutritional value of a specific food.
Methodology:
This application took around three months to develop. The team went through various phases of development which required consultation and collaboration.
We come from two different tracks: AI and web development. We faced various hurdles; initially, we had five members, but two left. The member working on Fronted using React encountered technical problems and could not work for around one month.
For us working on web development, as we were beginners, we used all the resources offered to us on Edyoucated. We also combined courses from various sources such as Udemy, YouTube, and more. Given the nature of the project, we felt it was necessary to learn Full Stack web development. The course Zero to Mastery helped us a lot doing that. Moreover, we went beyond what was offered; it was crucial to make use of all resources available online (courses on Youtube helped as well). The training beyond the official courses helped us as well to learn more about React native, Expo, and Firebase.
For us, the AI team, we exhausted the whole learning path on Edyoucated. Moreover, beyond that, we learned, for example, on YouTube. During the learning period, we would learn many skills and little acquirements that would come in very handy for our project.
When we started working on the AI, we encountered a significant problem. There is almost no database containing pictures of various food items. During the Fastai course, we luckily learned how to establish such a database with the help of azure. Thus, we went on using azure to download pictures from Bing.
By establishing our own database, we encountered one of the main aspects of creating a good database, which is mainly having useful and clean data. Therefore, most of our time was used to clean the data. After a few weeks, we had our database settled and were finally able to start developing our AI. Right from the beginning, we knew that it would be nearly impossible to create a valuable AI by ourselves. Hence, we decided to use the resnet34, which is a 34 layer convolutional neural network.
Given that some of us also work on AI at our Universities and during our internship, we got additional training outside. In the end, despite the problems we encountered and thanks to these skills, we succeeded using the following to complete the project Angular for Frontend, .Net, and MongoDB for the backend, Fastai, Jupyter Notebooks, and Flask for the AI.
Finally, it was also crucial to managing workflow and collaboration. Thus, we made use of some concepts and working methods we got trained on throughout the program. For example, we used the Agile methods and developed Gantt charts within that framework. The talks during the workshops were also helpful as these were important interactions sources of precious counsels on how to work within a team.
Results:
At the end of our journey, we succeeded in building an application allowing users to register, sign in, upload picture(s), and get the nutritional value data of the food they want to get information on.
This is a work in progress, and it is not yet an application that can be fully deployed. As a next step, we intend to train more data so that our outputs are more precise, and our image classifier is fast to identify images with very high precision and accuracy. Moreover, we would like to improve the frontend of the application and change the upload functionality of the picture into a camera. This is more practical for users as they will not have to search for pictures on their cellphones and upload, they will only take a picture, and this will be sent to our database right away to get the nutritional value data.
As a long-term goal, we would like to do a full project out of this endeavor and fully deploy the application. For that, we would need funding, human resources, and infrastructure.
GitHub repository:
Team Members:
Artificial Intelligence:
- Alexander Lupatsiy [lupatsiy.alex@gmail.com, responsible for Artificial Intelligence]
- Niklas Stahl [niklasstahl98@gmail.com, LinkedIn: Niklas S, responsible for frontend, backend and Artificial Intelligence]
Web Development:
- Velomahanina Tahinjanahary Razakamaharavo [VeloRazaka@protonmail.com, responsible for frontend development]
Team Mentors:
- Florian Zimmer, Project Manager TechLabs
- Tom Stein, Web Development Mentor
- Kai Bitterschulte, Artificial Intelligence Mentor
