Project information
Project title: Development of quality control and management solution for the food industry based on computer vision and data science.
Project number: 1.2.1.1/18/A/002
Short description: The study is an interdisciplinary project between the largest food production company in Latvia JSC Dobeles Dzirnavnieks and the computer vision-based production quality developer SIA Zippy Vision with the aim to solve the product quality control challenges identified and installed by JSC Dobeles Dzirnavnieks, which can be applied in a wider range of industries.
The aim of the research is to create a computer vision and production management solution to improve innovative quality control based on the results of existing research. This includes the development of a unique product tracking system that would allow the collection of production information and quality inspection visual information at the product item level, including keeping images throughout the product's shelf life.
The solution can be used to manage complaints, ensure full product tracing for customers and continuously improve the quality control system using the accumulated data. The target market for this project is food producers who plan to significantly improve quality control or for whom such improvements are required by the market. The project is based on the expanded use of deep neural network technologies for computer vision and production management solution classification and inspection, in combination with the use of various types and technologies of industrial computer vision cameras in continuous production processes.
About the Competence Center.
The research is implemented within the project Latvian Food Industry Competence Center.
The Latvian Food Industry Competence Center covers two scientific activities directions: Increasing the market of Latvian producers' products, which includes:
1. Innovative products and technologies, following the latest in the circular economy trends (including eco-technological products, food safety and quality monitoring solutions);
2. Product packaging solutions;
3. Food logistics solutions, incl. technologies that improve shelf-life and reduce transport costs, while maintaining high product quality and shortening the journey from producer to consumer.
4. Increasing the added value and competitiveness of production, which includes: Improving the use of food by-products in higher value-added products.
5. More appropriate agricultural and fruit research to improve resource efficiency - Species, cultivation technologies;
6. New, innovative products with higher added value by making maximum use of existing markets, technologies, other production factors.
The research project is carried out within the framework of the Latvian Food Industry Competence Center (LPKC). The aim of LPKC is to support at least 20 researches in accordance with the defined research directions, developing new products and technologies in the sector and introducing it into production, thus promoting cooperation between the research and industry sector, as well as increasing the competitiveness of the industry.
The project is implemented from April 8, 2019 to December 31, 2021.
The total amount of ERDF support to the Competence Center is EUR 4,708,111.50.
Progress updates
June 30, 2019
Developed and stabilized image acquisition and processing methodology using cameras, sensors and intelligent lighting. Developed data processing tools for neural network training.
August 30, 2019
Robust, font and language agnostic label printing control technology is developed
November 16, 2019
The exact final problem has been confirmed and verified.
The list of technologies to be used has been strengthened, which would allow not to duplicate the solutions available on the market.
Analyzed and verified list of solutions at international food producers.
Hypotheses for all solutions of technological challenges (problem formulation) to be solved and verified.
75% of the set of technological parameters is obtained technology for robust image acquisition technology based on light filters, cameras. Developed control software that controls lighting, image acquisition parameters.
Improved label printing control technology that is font and language agnostic.
Semi-developed technologies for automated data acquisition, augmentation and classification on the user side in a way that maximally simplifies the adaptation of solutions based on deep neural networks
An annotation platform has been created, which is connected to the device's image acquisition system.
The objectives of the annotation platform are:
- quality assessment;
- manual review of quality inspection session data;
- annotation of new defects with the aim of improving deep learning models.
An annotation platform is a cloud-based system using data previously generated by the device and information from the device's physical sensors. Partially developed system for automated data transfer to the annotation system from the device for annotation and quality assessment purposes
A deep learning model for the assessment of glueing defects has been developed. The model needs to supplement the training data with the control of transparent sealing defects, taking into account the tolerance for insignificant deviations.
Improved print quality (label print) inspection algorithm taking into account that the print length is variable in the direction of tape movement. The new algorithms use operations to estimate the number of print missing relative to a reference image.
A faster label positioning system has been created.
Obtained packet processing speed up to 150 ms from 700ms.
This allows the rejector to be placed directly on the machine. A new modular solution deployment system has been developed to ensure a limited allocation of resources to each of the services, to ensure service autonomy and a unified communication protocol. The new deployment system allows you to isolate each subsystem, ensuring that each subsystem can use its own operating system and software dependencies.
Migrated to Linux to provide GPU resource sharing between services on a single computer.
February 16, 2019
August 16, 2020
Obtained solutions for speed improvements of the existing technology for the inspection up to 700 packages per minute and for line speeds up to 4 m / s.
Obtained solutions for reducing the size limitations of the test object by changing the image transmission technologies and processing system. An additional system with a camera on the flow-wrap packaging film, after the printer, has been developed and started to be tested, for a more accurate evaluation of the print quality, ensuring full testing of the print located on the edge or end of the package.
Centralized administration functions have been established, incl. Development of API for coupling with a centralized production management system
November 11, 2020
Izveidots produktu izsekošanas prototips un kontatētas pielietojamības robežas
Izstrādātas centralizētas vadības tehnoloģijas inegrācijai vadības sistēmās
Pilnveidota marķēšanas plūsma jaunu produktu un attēlu marķēšanai un neironu tīklu trenēšanai
Iegūtas tehnoloģijas ātri kustīgu objektu detektēšanai – pieteikts patents.
Tiek pabeigts integrēta risinājuma prototips
Anotēšanas platforma un dziļā mācīšanās:
- kvalitātes novērtējums, manuāli pārskatot kvalitātes inspekcijas sesijas datus, jaunu defektu anotēšana ar mērķi uzlabot dziļās mācīšanās modeļus
- Anotēšanas platforma ir mākoņskaitļošanā bāzēta sistēma izmantojot iepriekš iekārtas ģenerētos datus un iekārtas fizisko sensoru informāciju
- Pabeigta sistēma automatizētai datu pārnešanai uz anotēšanas sistēmu no iekārtas, anotēšanas un kvalitātes vērtēšanas vajadzībām
- Uzlabots dziļo tīklu modelis līmējumu defektu novērtēšanai. Modelis papildināts ar treniņa datiem ar caurspīdīgo līmējumu defektu kontroli, ņemot vērā toleranci pret nenozīmīgiem defektiem
Atsevišķas kameras ieviešana aiz apdrukas
Iespēja precīzi noteikt apdrukas kvalitāti uz lentes – jebkurā apdrukas vietā
Pilnveidots drukas kvalitātes (etiķetes drukas) novērtēšanas algoritms ņemot vērā, ka lentes kustības virzienā drukas garums ir mainīgs. Jaunie algoritmi izmanto attēla morfoloģijas darbības lai novērtētu punktu daudzumu, kas trūkst attiecībā pret etalonu izmantojot konvolūcijas filtrus.
Projektā iegūtās tehnoloģijas veiksmīgi tiek ieviestas plašākā industriju spektrā – kokmateriāli un celtniecības materiāli.