← العودة إلى الجلسة العربيةSESSION 04 · ENGLISH COMPANION

Computer Vision Engineering

From pixels to measurable decisions. The Arabic edition includes the schedule, deliverables and grading criteria.

Why This Session Is Different#

In real-world computer vision, running YOLO and drawing bounding boxes is only the beginning. A computer vision engineer must transform model predictions into reliable, meaningful, measurable outputs while understanding system limitations and failure modes.

This session addresses one concrete engineering problem:

Instead of two disconnected beginner projects, students build one coherent video analytics system: from raw video to structured crossing events, summary metrics, an annotated video, and an evaluation report.

Learning Outcomes#

By the end of the session, participants will be able to:

  • Distinguish image classification, object detection, and multi-object tracking, and explain when each is needed.
  • Explain the effects of video resolution, frame rate, confidence thresholds, and IoU on accuracy and speed.
  • Run a pretrained object detector on video and interpret its outputs correctly.
  • Describe how a tracker associates objects across frames and why track IDs can change.
  • Turn object trajectories into real crossing events, rather than counting every detection as a new vehicle.
  • Apply direction-aware line crossing and duplicate-count prevention rules.
  • Export machine-readable CSV/JSON data alongside an annotated demonstration video.
  • Measure counting error and processing throughput, and analyze at least three realistic failure cases.
  • Deliver a reproducible experiment with basic tests and documentation.

Part I — How a Computer Vision Engineer Thinks#

1. Problem Framing: From Business Request to Measurable Requirements#

Scenario: A transportation operations team wants to estimate vehicle flow through a defined checkpoint without continuous manual counting.

QuestionEngineering Decision
What is the desired output?Vehicle crossing totals by direction and class
What is the input?Fixed-camera video with known dimensions and FPS
What is the unit of measurement?One valid crossing event, not one bounding box
Which categories matter?Supported detector classes such as car, bus, truck, motorcycle
What can fail?Occlusion, congestion, camera shake, poor lighting, ID switches
How will accuracy be checked?Compare predicted counts/events against a manually annotated reference clip

2. Video Fundamentals#

  • Images as pixel arrays, color channels, and resizing operations.
  • Resolution: the trade-off between computational cost and visibility of small objects.
  • FPS, frame index, and timestamp: source-video time versus processing throughput.
  • Reading video and writing annotated outputs with OpenCV.
  • Region of Interest (ROI) and counting lines: configuring spatial rules in a particular video coordinate system.

3. Object Detection#

  • Brief review of CNNs and transfer learning as a continuation of Session 03.
  • Interpreting bounding boxes, class IDs, and confidence scores.
  • Intersection over Union (IoU) and the conceptual role of Non-Maximum Suppression.
  • How changing the confidence threshold affects downstream detections.
  • Model-size, accuracy, and latency trade-offs for the available hardware.

4. Multi-Object Tracking#

  • Detection: What object is visible, and where is it in the current frame?
  • Tracking: Does this detection correspond to an object seen in previous frames?
  • Track IDs, trajectories, and temporal association.
  • Practical overview of ByteTrack and BoT-SORT.
  • Failure modes: ID switches, temporary occlusion, lost tracks, and reappearing objects.

5. Event Engineering: From Object Tracks to Useful Information#

The most important conceptual step is converting raw model outputs into application logic:

  1. Define a virtual counting line.
  2. Select an appropriate reference point for each tracked vehicle, such as the bottom-center of its bounding box.
  3. Track which side of the line the point occupies over time.
  4. Determine crossing direction.
  5. Emit a crossing event only when valid transition conditions are satisfied.
  6. Use state and hysteresis to reduce repeated counts caused by jitter near the line.
  7. Prevent duplicate events for a track while acknowledging that ID switches can still create duplicate counts.
MetricMeaning
OccupancyNumber of objects currently present inside a defined region
Crossing CountCumulative number of objects crossing a line or boundary
Unique Track CountNumber of track identifiers, which may not equal the true number of distinct vehicles

Core Project — Traffic Flow Intelligence#

Product Concept#

Build the core of a traffic video analytics application that processes one fixed-camera clip, detects and tracks vehicles, records directional crossing events, and produces outputs that could later feed a dashboard or API.

A full web UI is intentionally out of scope. This session prioritizes correct computer vision logic, structured results, and measurable system quality.

System Architecture#

flowchart TD
    A["Video Input"] --> B["Frame Decode & Timestamp"]
    B --> C["Object Detector"]
    C --> D["Multi-Object Tracker"]
    D --> E["Line Crossing & Event Rules"]
    E --> F["Events CSV + Summary JSON"]
    E --> G["Annotated Video"]
    F --> H["Evaluation Against Ground Truth"]
تدفق النظام كما ورد في منهج الجلسة

Prepared Project Inputs#

  • Video A: A short, clear fixed-camera traffic clip for baseline implementation.
  • Video B: A harder clip with occlusion, vehicle overlap, or lighting changes for failure analysis.
  • A lightweight pretrained detector and its model weights, downloaded in advance.
  • A configuration file containing the input path, counting line, target classes, and thresholds.
  • A small ground-truth reference: manually verified crossing counts by class and direction for the evaluation clip.

Stage 1 — Video Ingestion & Baseline Detection#

  • Open the video and verify frame dimensions and source FPS.
  • Load the pretrained detection model.
  • Run inference on sample frames.
  • Visualize boxes, class labels, and detection confidence.
  • Select relevant vehicle classes and store the experiment configuration.

Stage 2 — Multi-Object Tracking#

  • Enable tracking across consecutive frames.
  • Visualize track IDs and inspect whether an object keeps its ID across time.
  • Test an occlusion or overlap case and observe ID stability.
  • Separate detection failures from identity-association failures.

Stage 3 — Business Event Logic#

  • Configure a counting line and define both travel directions.
  • Compute a vehicle reference point and track which side of the line it occupies.
  • Record a crossing only after a valid directional transition.
  • Implement duplicate suppression and test jitter near the line.
  • Store event metadata: timestamp, object class, direction, track ID, and video ID.

Illustrative crossing-event record (example values only):

json
{
  "video_id": "intersection_demo_01",
  "timestamp_seconds": 12.4,
  "track_id": 17,
  "class_name": "car",
  "direction": "in",
  "event_type": "line_crossing",
  "line_id": "main_entry"
}

Stage 4 — Reporting & Visual Evidence#

  • Export events.csv containing individual crossing events.
  • Generate summary.json with totals per vehicle class and direction.
  • Produce annotated.mp4 with bounding boxes, track IDs, counting line, and running totals.
  • Provide a clear script or command-line entry point for reproducible execution.

Stage 5 — Evaluation & Failure Analysis#

Evaluate the end-to-end system, not just the detector's confidence scores.

Evaluation AreaRequired Check
Count ErrorCompare predicted and manually verified counts per direction
Duplicate EventsCheck for repeated events near the counting line
ThroughputMeasure processed frames per second on the test machine
Failure CasesRecord examples involving occlusion, ID switches, or missed detections
ReproducibilityRerun the same video and configuration with documented dependencies

An accessible first metric is absolute counting error:

Ecount=∣Npredicted−Nreference∣E_{\mathrm{count}}=\left|N_{\mathrm{predicted}}-N_{\mathrm{reference}}\right|

Basic Tests for Event Logic#

  • An object crosses the line once and generates exactly one event.
  • An object approaches the line and turns back without generating an event.
  • A reference point jitters near the line without creating repeated events.
  • Two separate objects cross and produce two valid events.

Instructor Preparation Checklist#

  • Select two traffic videos with appropriate usage permissions: one easy and one challenging.
  • Create a manually checked ground-truth reference for the evaluation clip.
  • Download lightweight model weights and pin dependency versions.
  • Test a CPU-only fallback in case student GPUs are unavailable.
  • Prepare a starter repository with video loading and configuration code.
  • Provide at least two isolated tests for crossing-event logic.
  • Keep an annotated backup demo video for environment or timing failures.

Optional Extensions After Class#

For advanced participants or a longer capstone:

  1. Compare ByteTrack and BoT-SORT on the difficult video.
  2. Support multiple ROIs and report current occupancy separately from cumulative crossing counts.
  3. Measure how resizing or skipping frames affects count error and FPS.
  4. Provide a small API that serves summary.json without rerunning the model on each request.
  5. Collect a task-specific dataset and perform fine-tuning only when error analysis establishes a need.

Official References#


Session Takeaway#

This session teaches students that professional computer vision is more than running a pretrained model. It means defining a measurable problem, combining detection with tracking and temporal logic, producing useful structured data, and evaluating the system against real failure cases.

ما الذي تريد تعلّمه؟

بحث في محتوى الجلسات العشر. استخدم Esc للإغلاق.