Giters

bsm8734 / real-time-hand-tracking

πŸ₯ Real-time hand-tracking to play musical instruments

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

csrtopencvpythonssd

Real-time hand-tracking

  • 2019-1 ν•™κΈ° μΈν•˜λŒ€ν•™κ΅ λ©€ν‹°λ―Έλ””μ–΄μ»΄ν“¨νŒ… 과제(ν”„λ‘œμ νŠΈ) μˆ˜ν–‰μ„ κΈ°λ‘ν•œ λ ˆν¬μž…λ‹ˆλ‹€.
  • ν”„λ‘œμ νŠΈ κ²°κ³Όλ³΄κ³ μ„œλŠ” μ†Œλ…Όλ¬Έ ν˜•μ‹μœΌλ‘œ μž‘μ„±λ˜μ—ˆμœΌλ©°, μ—¬κΈ°μ—μ„œ ν™•μΈν•˜μ‹€ 수 μžˆμŠ΅λ‹ˆλ‹€.

μΆ”ν›„, 이 μ†ŒμŠ€μ½”λ“œλ₯Ό ν™œμš©ν•˜μ—¬ 2019 Problem Solving κ²½μ§„λŒ€νšŒμ—μ„œ
λ“œλ‘ μ„ ν™œμš©ν•œ 읡수자 좔적 μ†Œν”„νŠΈμ›¨μ–΄ μ œμž‘μ— κΈ°μ—¬ν•˜μ˜€μœΌλ©°,
πŸ…κΈˆμƒμ„ μˆ˜μƒν•˜μ˜€μŠ΅λ‹ˆλ‹€.

μˆ˜ν–‰μΌμž

  • ν”„λ‘œμ νŠΈ μˆ˜ν–‰κΈ°κ°„: 2019.03. ~ 2019.06.
  • 곡λͺ¨μ „ μˆ˜ν–‰κΈ°κ°„: 2019.07.

ν”„λ‘œμ νŠΈ μ„€λͺ…

  • μ‹€μ‹œκ°„μœΌλ‘œ 물체(손)을 μΆ”μ ν•˜μ—¬, ν”„λ ˆμž„ λ‚΄ 일정 ꡬ간에 μ§„μž…ν–ˆμ„ λ•Œ, 타악기 μ†Œλ¦¬λ₯Ό λ‚΄λŠ” ν”„λ‘œκ·Έλž¨
  • λ”₯λŸ¬λ‹ 방식인 SSD(Shot Multi Box Detector)와 openCV의 CSRT Trackerλ₯Ό μ‚¬μš©ν•˜μ—¬ κ΅¬ν˜„
  • μ‹€μ‹œκ°„μ„±μ˜ 기쀀을 20 FPS둜 μ„€μ •

μ•„ν‚€ν…μ²˜

SSD (Single Shot Multi Box Detector)

CSRT (Discriminative Correlation Filter with Channel and Spatial Reliability)

  • [paper] Discriminative Correlation Filter with Channel and Spatial Reliability

  • // μž‘μ„±μ€‘

  • μ§€μ •λœ λ°•μŠ€ μ’Œν‘œλ₯Ό λ„˜κ²¨λ°›μ•„, λ°•μŠ€ μ•ˆμ˜ ν”½μ…€ 이동을 νŒŒμ•…ν•˜λŠ” λ°©λ²•μœΌλ‘œ 물체λ₯Ό μΆ”μ ν•˜λŠ” 역할을 ν•©λ‹ˆλ‹€.

  • 기쑴의 Single-Channel CF trackerμ—μ„œ channel reliability와 spatial reliabilityλ₯Ό λ„μž…ν•œ λ°©λ²•μž…λ‹ˆλ‹€.

  • Single-Channel CF tracker

  • Channel reliability

  • Spatial reliability

  • OpenCVμ—μ„œ μ œκ³΅ν•˜λŠ” tracker 쀑 ν•˜λ‚˜

    • OpenCVμ—μ„œ μ œκ³΅ν•˜λŠ” tracker인 KCF, CSRT, MOSSE λ“±μ˜ 트레컀λ₯Ό μ‚¬μš©ν•˜μ˜€λŠ”λ°, 각 트레컀 λ§ˆλ‹€ μ„±λŠ₯κ³Ό μ†λ„μ˜ 차이가 μ‘΄μž¬ν–ˆμŠ΅λ‹ˆλ‹€.
    • νŠΉμ§•
      • CSRT trackerλŠ” μƒλŒ€μ μœΌλ‘œ 높은 정확도λ₯Ό λ³΄μ˜€μœΌλ‚˜, ν”„λ ˆμž„μ΄ 16 ν”„λ ˆμž„ μ•ˆνŒŽ μ •λ„λ‘œ νŠΈλ ˆν‚Ήν•˜λŠ”λ° 어렀움을 κ²ͺ음
      • MOSSE trackerλŠ” ν‰κ· μ μœΌλ‘œ 100ν”„λ ˆμž„ μ΄μƒμ˜ 높은 ν”„λ ˆμž„μ„ μžλž‘ν–ˆμœΌλ‚˜, μ €μ‘°ν•œ μ •ν™•λ„λ‘œ μΈν•˜μ—¬ νŠΈλ ˆν‚Ή ν•˜λŠ” 물체λ₯Ό 자주 놓침 image

μˆ˜ν–‰ν™˜κ²½

λͺ¨λ“  μ‹€ν—˜μ€ μ•„λž˜μ˜ ν™˜κ²½μ—μ„œ μˆ˜ν–‰λ˜μ—ˆμœΌλ©°, μ΄λŸ¬ν•œ 저사양 κΈ°κΈ°μ—μ„œλ„ μˆ˜μ›”ν•˜κ²Œ μ‹€μ‹œκ°„ μ—°μ£Όκ°€ κ°€λŠ₯함을 λ³΄μ˜€μŠ΅λ‹ˆλ‹€.

  • CPU: Intel I3-5005U (2.00GHz)
  • Ram 4GB
  • Graphic Intel HD graphics 5500 (λ…ΈνŠΈλΆ λ‚΄μž₯ κ·Έλž˜ν”½μΉ΄λ“œ)

About

πŸ₯ Real-time hand-tracking to play musical instruments

csrtopencvpythonssd

Languages

Language:Python 74.2%Language:Starlark 25.8%