Crowd Estimation at Green Spaces
While Singapore’s Circuit Breaker has lifted, safe distancing may remain a need for the foreseeable future, as we continue to slow the spread of the coronavirus. During this period, authorities will need to continue to ensure safe distancing in public spaces, such as parks, gardens and nature reserves.
Since the initial onset of the pandemic, the National Parks Board (NParks) has enforced safe distancing measures at all its parks, gardens and nature reserves, including Park Connectors, Pulau Ubin and parks managed by town councils. This includes crowd estimation and park patrols at these green spaces. This can be manpower intensive, especially for large public spaces, not to mention laborious when compiling and integrating data into the Safe Distance @ Parks portal.
This is where technology such as drones (UAVs) and Computer Vision (CV) excel. Put together, they provide superior speed, scope, and scalability over manual surveys. NParks has used drones to supplement their efforts in monitoring visitorship levels in major parks, gardens and nature areas, and recognizes that drones with real-time analytics can be an additional tool for crowd counting.
Safe Distance @ Parks is an interactive portal for the public to look for a park near them and check visitorship levels before leaving their house.
Different coloured symbols – an orange square, yellow diamond and green circle – are used to represent the level of visitorship in a specific park.
For example, if an area is experiencing high visitorship, which is indicated by an orange square, the public is advised to refrain from visiting it.
Hackathon at Garuda Robotics
Back in March when we were told to stay home, and half our business postponed overnight, we rallied our people around what we do best: building the future of drone technologies. Except this time, it was a different future, a future that requires drones to play a central role in public health. We split ourselves into 3 teams, and I was part of the team who investigated how we could deploy our AI platform to help NParks estimate crowd levels in parks, gardens and nature reserves.
Our goal was to go beyond the 3 tiers published on Safe Distance @ Parks, and to provide fairly accurate crowd counts in real time from UAVs. Under the Government’s COVID-19 regulations, our company was not regarded as performing an essential service. As such, we drew up a plan where NParks would continue to perform the patrolling, while tapping on our technology to use their drones for crowd estimation. This will complement ground observations and give them a better sense of the density of visitors in a specific area.
The constraints of the system match closely with many similar people counting applications:
- No personally identifiable information should be captured, i.e. there is only enough information to recognize the shape of a human but not enough to know who.
- The goal of the algorithm is to count the number of people over a given flight without double counting as people move around.
- The output of the system that are stored and processed are only people counts, all raw videos are discarded after the count is obtained.
Deep Learning Approach
To work within the constraints, let us take a look at our workflow and algorithm to this challenging problem.
Workflow
- We estimate there to be several drones flying concurrently across various parks, gardens and nature areas at any one time, especially during morning and evening peak hours.
- Each drone is capable of streaming video live to a secure AI platform in the cloud.
- This AI Platform will sanitize the footage and estimate the number of unique persons throughout the park.
- By the end of the flight, the system will report the values to Safe Distance @ Parks administrators to be incorporated into the portal.
Example in-flight footage from step 3
Algorithm
- For object detection, we run our proprietary people detection models previously trained on aerial footage from computer vision research repositories. This generates bounding boxes as above.
- To remove duplicates, we run a well-known deep learning algorithm (DeepSORT) that generates a collection of 128-dimension embeddings for each bounding box that’s matched against embeddings on subsequent video frames.
The pipeline assumes every start / stop of the live stream to be 1 flight and provides people count summary on a flight by flight basis to the appropriate API end points for further summarizing.
Real World Implementation and Operationalization
A good algorithm is not that useful if it only works on a developer or researcher’s computer. By choosing to run the system on our AI platform, NParks or similar customers have the option of deploying this proof of concept on the cloud, on premise, or even on powerful edge devices with GPUs.
Fortunately, we already have the capability to overcome the major hurdles. For example, we can sustain a usable video stream from the drone over mobile networks, whether it’s via our handheld GCS mobile app Plex Pilot, or an on-board computer Garuda CoPilot with dual networks (2 SIM Cards).
Other considerations include the systems integration work necessary between the principal and these autonomous systems and their respective C2 and data analytics platforms.
Next Steps
We successfully emerged from our hackathon and delivered on our very first prototype to a delighted customer. While we won’t know how it would eventually be useful at this juncture, we’re confident that we can take this forward to enhance the product into a more multifaceted product for public health at green areas.
We also trained our model to count cars too, perhaps an LTA project next?
So what do you think? What else can we do in these trying times, to help the country recover sooner from the lockdown? Do share your thoughts with us.
Thank you.