Serket is a computer vision project using blob detection methods to recognize different objects in the camera’s field of view. The project was developed for demonstration purposes based on the original Serket product, which uses computer vision in real-life industrial pig farming. The camera is pointing at a mockup of a pig barn and my task was to detect the presence of plastic pigs along with their orientation and position.
Originally a web developer then transitioning into art and technology installations, I never had much to do with computer vision before I got this project. The time was pressing as I had only two months to both learn computer vision and develop a functioning prototype. I worked closely together with Bálint Zsiga, whose already extensive knowledge in computer vision was of great help. Bálint and I followed a similar path into programming, in which we learned through a problem-based approach where we seek out solutions to a current problem we have, omitting the institutionalised approach to learning.
Although I had some programming courses in university, the educational institution I attended utilized the problem-based learning pedagogy, in which students solve an open-ended problem by an inquiry rooted in social constructivism, where knowledge is constructed by interaction with other students (Aalborg Universitet, 2015). This approach can also be recognized in online knowledge-making behavior, where individuals cluster around areas of interest.
Online code sharing platforms and message boards make it possible for anyone to learn, teach and contribute to software projects, and we relied on them heavily during the development of Serket.
The development process was characterized by the use of two open-source software library projects: Processing and OpenCV. The communities contributing to these projects are built around knowledge sharing and are not driven by financial aims — although many professionals use them for their paid projects, the advice is available for free and is based on a shared effort to make programming an available skill for everyone, independent of their social, economic or cultural background. This connectedness means that whenever someone somewhere wants to create a project which requires programming, he or she has the knowledge available and can develop his or her idea either alone or with the help of the community. Within such communities, individuals interact with each other as social beings, rather than as market actors (Gauntlett, 2018).
In order to illustrate how this learning approach works in practice, I am going to take a look at the software which I used at the beginning of the project, and the community which helped me learn and understand the foundations of computer vision. Processing is a programming sketchbook and a development tool mainly for the visual arts, with software prototyping and data visualization as its two most widespread areas of usage. Although the final prototype did not run in Processing, the software was exceptionally useful during the first half of development, as I could quickly and easily sketch out different ideas while investigating the possible approaches to blob detection. The Processing Foundation’s vision is to make programming available for everyone, therefore the software is developed by volunteers and is distributed freely. The syntax is developed so that people coming from a background other than computer science can easily get the hang of it and start producing functional sketches right away (Processing Foundation, 2020). Examining the case of Processing, we can clearly see that knowledge is constructed in an active manner resulting from interactions among individuals, bypassing gatekeepers and institutionalized learning.
I learned the basics of computer vision such as colour tracking and motion detection from the Processing community, where creative coders use their free time to share their knowledge. Once I understood the basics, I integrated a software library which was specifically developed for computer vision and which made the development process faster and somewhat easier. OpenCV stands for open-source computer vision and unlike Processing, major companies are core contributors to the project in order “to provide a common infrastructure for computer vision applications and to accelerate the use of machine perception in the commercial products.” (OpenCV team, 2019)
The community still plays a large role in pointing out shortcomings, making improvements and shaping new features. Since OpenCV is a community-driven project, there are countless tutorials and code snippets available for free, which helped us develop a functioning prototype in such a short time.
From society’s perspective, open-source is the most economically efficient solution, where those who possess information give it away for free, or rather for the cost of communicating it. The resulting projects are openly shared and available for others to build on, extend, and make their own. Benkler (2006) calls this phenomenon “the networked information economy”, where “all the inputs necessary to effective productive activity are under the control of individual users.” (p. 99)
As written before, Bálint and I followed a problem-based development approach which was also due to the time limitation. As computer vision is a huge field, one has to gain a certain level of familiarity with the library first in order to be able to think of possible solutions to a problem.
Although the technology behind Serket was quite challenging for us and we went through many trials and errors, we never faced a problem which somebody else has not already had, and our task was to utilize algorithms already integrated into the library and follow advice from the OpenCV community. We did not use any of the technology from the original Serket project but instead developed our own, tailored specifically for the mockup. This means that our development process was heavily based on quick prototyping and we chose our methods of inquiry accordingly. Our approach may not be suitable for larger projects where specialist knowledge in a particular field is necessary, however, in the case of a project where one has to quickly show results without deep prior knowledge in the field, open-source software and most importantly its surrounding communities are the best place to turn to when it comes to knowledge-making and problem-solving.
Aalborg Universitet. (2015). Problem Based Learning . Retrieved from https://www.aau.dk/digitalAssets/148/148025_pbl-aalborg-model_uk.pdf
Benkler, Y. (2006). The wealth of networks: how social production transforms markets and freedom. New Haven, CT: Yale University Press.
Gauntlett, D. (2018). Making is Connecting: the social power of creativity, from craft and knitting to digital everything (2nd ed.). Cambridge: Polity.
OpenCV team. (2019). Retrieved January 23, 2020, from https://opencv.org/about
Processing Foundation. (2020). Retrieved January 23, 2020, from https://processingfoundation.org/