Why This Session Is Different#
In real-world computer vision, running YOLO and drawing bounding boxes is only the beginning. A computer vision engineer must transform model predictions into reliable, meaningful, measurable outputs while understanding system limitations and failure modes.
This session addresses one concrete engineering problem:
Instead of two disconnected beginner projects, students build one coherent video analytics system: from raw video to structured crossing events, summary metrics, an annotated video, and an evaluation report.
Learning Outcomes#
By the end of the session, participants will be able to:
- Distinguish image classification, object detection, and multi-object tracking, and explain when each is needed.
- Explain the effects of video resolution, frame rate, confidence thresholds, and IoU on accuracy and speed.
- Run a pretrained object detector on video and interpret its outputs correctly.
- Describe how a tracker associates objects across frames and why track IDs can change.
- Turn object trajectories into real crossing events, rather than counting every detection as a new vehicle.
- Apply direction-aware line crossing and duplicate-count prevention rules.
- Export machine-readable CSV/JSON data alongside an annotated demonstration video.
- Measure counting error and processing throughput, and analyze at least three realistic failure cases.
- Deliver a reproducible experiment with basic tests and documentation.
Part I — How a Computer Vision Engineer Thinks#
1. Problem Framing: From Business Request to Measurable Requirements#
Scenario: A transportation operations team wants to estimate vehicle flow through a defined checkpoint without continuous manual counting.
| Question | Engineering Decision |
|---|---|
| What is the desired output? | Vehicle crossing totals by direction and class |
| What is the input? | Fixed-camera video with known dimensions and FPS |
| What is the unit of measurement? | One valid crossing event, not one bounding box |
| Which categories matter? | Supported detector classes such as car, bus, truck, motorcycle |
| What can fail? | Occlusion, congestion, camera shake, poor lighting, ID switches |
| How will accuracy be checked? | Compare predicted counts/events against a manually annotated reference clip |
2. Video Fundamentals#
- Images as pixel arrays, color channels, and resizing operations.
- Resolution: the trade-off between computational cost and visibility of small objects.
- FPS, frame index, and timestamp: source-video time versus processing throughput.
- Reading video and writing annotated outputs with OpenCV.
- Region of Interest (ROI) and counting lines: configuring spatial rules in a particular video coordinate system.
3. Object Detection#
- Brief review of CNNs and transfer learning as a continuation of Session 03.
- Interpreting bounding boxes, class IDs, and confidence scores.
- Intersection over Union (IoU) and the conceptual role of Non-Maximum Suppression.
- How changing the confidence threshold affects downstream detections.
- Model-size, accuracy, and latency trade-offs for the available hardware.
4. Multi-Object Tracking#
- Detection: What object is visible, and where is it in the current frame?
- Tracking: Does this detection correspond to an object seen in previous frames?
- Track IDs, trajectories, and temporal association.
- Practical overview of ByteTrack and BoT-SORT.
- Failure modes: ID switches, temporary occlusion, lost tracks, and reappearing objects.
5. Event Engineering: From Object Tracks to Useful Information#
The most important conceptual step is converting raw model outputs into application logic:
- Define a virtual counting line.
- Select an appropriate reference point for each tracked vehicle, such as the bottom-center of its bounding box.
- Track which side of the line the point occupies over time.
- Determine crossing direction.
- Emit a crossing event only when valid transition conditions are satisfied.
- Use state and hysteresis to reduce repeated counts caused by jitter near the line.
- Prevent duplicate events for a track while acknowledging that ID switches can still create duplicate counts.
| Metric | Meaning |
|---|---|
| Occupancy | Number of objects currently present inside a defined region |
| Crossing Count | Cumulative number of objects crossing a line or boundary |
| Unique Track Count | Number of track identifiers, which may not equal the true number of distinct vehicles |
Core Project — Traffic Flow Intelligence#
Product Concept#
Build the core of a traffic video analytics application that processes one fixed-camera clip, detects and tracks vehicles, records directional crossing events, and produces outputs that could later feed a dashboard or API.
A full web UI is intentionally out of scope. This session prioritizes correct computer vision logic, structured results, and measurable system quality.
System Architecture#
flowchart TD
A["Video Input"] --> B["Frame Decode & Timestamp"]
B --> C["Object Detector"]
C --> D["Multi-Object Tracker"]
D --> E["Line Crossing & Event Rules"]
E --> F["Events CSV + Summary JSON"]
E --> G["Annotated Video"]
F --> H["Evaluation Against Ground Truth"]Prepared Project Inputs#
- Video A: A short, clear fixed-camera traffic clip for baseline implementation.
- Video B: A harder clip with occlusion, vehicle overlap, or lighting changes for failure analysis.
- A lightweight pretrained detector and its model weights, downloaded in advance.
- A configuration file containing the input path, counting line, target classes, and thresholds.
- A small ground-truth reference: manually verified crossing counts by class and direction for the evaluation clip.
Stage 1 — Video Ingestion & Baseline Detection#
- Open the video and verify frame dimensions and source FPS.
- Load the pretrained detection model.
- Run inference on sample frames.
- Visualize boxes, class labels, and detection confidence.
- Select relevant vehicle classes and store the experiment configuration.
Stage 2 — Multi-Object Tracking#
- Enable tracking across consecutive frames.
- Visualize track IDs and inspect whether an object keeps its ID across time.
- Test an occlusion or overlap case and observe ID stability.
- Separate detection failures from identity-association failures.
Stage 3 — Business Event Logic#
- Configure a counting line and define both travel directions.
- Compute a vehicle reference point and track which side of the line it occupies.
- Record a crossing only after a valid directional transition.
- Implement duplicate suppression and test jitter near the line.
- Store event metadata: timestamp, object class, direction, track ID, and video ID.
Illustrative crossing-event record (example values only):
{
"video_id": "intersection_demo_01",
"timestamp_seconds": 12.4,
"track_id": 17,
"class_name": "car",
"direction": "in",
"event_type": "line_crossing",
"line_id": "main_entry"
}Stage 4 — Reporting & Visual Evidence#
- Export
events.csvcontaining individual crossing events. - Generate
summary.jsonwith totals per vehicle class and direction. - Produce
annotated.mp4with bounding boxes, track IDs, counting line, and running totals. - Provide a clear script or command-line entry point for reproducible execution.
Stage 5 — Evaluation & Failure Analysis#
Evaluate the end-to-end system, not just the detector's confidence scores.
| Evaluation Area | Required Check |
|---|---|
| Count Error | Compare predicted and manually verified counts per direction |
| Duplicate Events | Check for repeated events near the counting line |
| Throughput | Measure processed frames per second on the test machine |
| Failure Cases | Record examples involving occlusion, ID switches, or missed detections |
| Reproducibility | Rerun the same video and configuration with documented dependencies |
An accessible first metric is absolute counting error:
Basic Tests for Event Logic#
- An object crosses the line once and generates exactly one event.
- An object approaches the line and turns back without generating an event.
- A reference point jitters near the line without creating repeated events.
- Two separate objects cross and produce two valid events.
Instructor Preparation Checklist#
- Select two traffic videos with appropriate usage permissions: one easy and one challenging.
- Create a manually checked ground-truth reference for the evaluation clip.
- Download lightweight model weights and pin dependency versions.
- Test a CPU-only fallback in case student GPUs are unavailable.
- Prepare a starter repository with video loading and configuration code.
- Provide at least two isolated tests for crossing-event logic.
- Keep an annotated backup demo video for environment or timing failures.
Optional Extensions After Class#
For advanced participants or a longer capstone:
- Compare ByteTrack and BoT-SORT on the difficult video.
- Support multiple ROIs and report current occupancy separately from cumulative crossing counts.
- Measure how resizing or skipping frames affects count error and FPS.
- Provide a small API that serves
summary.jsonwithout rerunning the model on each request. - Collect a task-specific dataset and perform fine-tuning only when error analysis establishes a need.
Official References#
- Ultralytics — Multi-Object Tracking
- Ultralytics — Object Counting
- Ultralytics — Region Counting
- OpenCV Documentation
- Ultralytics — Licensing and Contributing
Session Takeaway#
This session teaches students that professional computer vision is more than running a pretrained model. It means defining a measurable problem, combining detection with tracking and temporal logic, producing useful structured data, and evaluating the system against real failure cases.
