Evaluation code

https://github.com/cheind/py-motmetrics

GitHub - cheind/py-motmetrics: :bar_chart: Benchmark multiple object trackers (MOT) in Python

MOT16 format

<frame>, <id>, <bb_left>, <bb_top>, <bb_width>, <bb_height>, <conf>, <x>, <y>, <z>

1280x720 

예시)

1,1,763.00,272.00,189.00,38.00,1,-1,-1,-1
1,2,412.00,265.00,153.00,30.00,1,-1,-1,-1
2,1,762.00,269.00,185.00,41.00,1,-1,-1,-1
2,2,413.00,267.00,151.00,26.00,1,-1,-1,-1
3,1,760.00,272.00,186.00,38.00,1,-1,-1,-1

위와 같은 형태로 gtSource.txt와 tSource.txt만들어야 함.

gt data : bboxes types velos accels ids

ids가 이상하게 되어있음....('-KpKb_7UDmsdQCf8FS3g9w', '1WP4TupVFubSnRO5xzftMw', '3EJCiJ8U4g4vbOqIcPAXMw', '83EloKWTYclrRJDFVF20bw', 'CgfQUOLj5xXGdRUGJ6qHpg', 'DBe_yG1gyQ6IhP0EzGgB0A', 'KQ1PvE1Mq7t4et5JWrjUIw')

association code를 살펴봐야 할 듯함.

Evaluation

def motMetricsEnhancedCalculator(gtSource, tSource):
  # import required packages
  import motmetrics as mm
  import numpy as np
  
  # load ground truth
  gt = np.loadtxt(gtSource, delimiter=',')

  # load tracking output
  t = np.loadtxt(tSource, delimiter=',')

  # Create an accumulator that will be updated during each frame
  acc = mm.MOTAccumulator(auto_id=True)

  # Max frame number maybe different for gt and t files
  for frame in range(int(gt[:,0].max())):
    frame += 1 # detection and frame numbers begin at 1

    # select id, x, y, width, height for current frame
    # required format for distance calculation is X, Y, Width, Height \
    # We already have this format
    gt_dets = gt[gt[:,0]==frame,1:6] # select all detections in gt
    t_dets = t[t[:,0]==frame,1:6] # select all detections in t

    C = mm.distances.iou_matrix(gt_dets[:,1:], t_dets[:,1:], \
                                max_iou=0.5) # format: gt, t

    # Call update once for per frame.
    # format: gt object ids, t object ids, distance
    acc.update(gt_dets[:,0].astype('int').tolist(), \
              t_dets[:,0].astype('int').tolist(), C)

  mh = mm.metrics.create()

  summary = mh.compute(acc, metrics=['num_frames', 'idf1', 'idp', 'idr', \
                                     'recall', 'precision', 'num_objects', \
                                     'mostly_tracked', 'partially_tracked', \
                                     'mostly_lost', 'num_false_positives', \
                                     'num_misses', 'num_switches', \
                                     'num_fragmentations', 'mota', 'motp' \
                                    ], \
                      name='acc')

  strsummary = mm.io.render_summary(
      summary,
      #formatters={'mota' : '{:.2%}'.format},
      namemap={'idf1': 'IDF1', 'idp': 'IDP', 'idr': 'IDR', 'recall': 'Rcll', \
               'precision': 'Prcn', 'num_objects': 'GT', \
               'mostly_tracked' : 'MT', 'partially_tracked': 'PT', \
               'mostly_lost' : 'ML', 'num_false_positives': 'FP', \
               'num_misses': 'FN', 'num_switches' : 'IDsw', \
               'num_fragmentations' : 'FM', 'mota': 'MOTA', 'motp' : 'MOTP',  \
              }
  )
  print(strsummary)