[Back-end] Darkflow-flask-web-streaming 구현

[Darkflow web streaming 기본 뼈대]

from darkflow.net.build import TFNet
import cv2

cap = cv2.VideoCapture('your_video_file')
model_path = "your_network_cfg.cfg"
weights_path = "your_network_weights.weights"

options = {"model": model, "load": weights_path, "threshold": 0.1, "gpu": 0.3}
tfnet = TFNet(options)
stop =False
while(not(stop)):
    ret, frame = cap.read()
  if(frame.size == 0):  #checking here for empty frame
    cap.release()    
    stop=True    
 result = tfnet.return_predict(frame)
 #DO SOMEHTING WITH RESULTS

[Darkflow web streaming]

- 전체 구조 (tree)

- flask server.py

from flask import Flask, url_for, render_template, Response
from darkflow.darkflow.net.build import TFNet
import cv2
import tensorflow as tf
import threading
import numpy


app = Flask(__name__)

options = {"model": "./darkflow/cfg/handlang-small.cfg",
           "pbLoad": "./darkflow/darkflow/built_graph/handlang-small.pb",
           "metaLoad": "./darkflow/darkflow/built_graph/handlang-small.meta" , "threshold": 0.4}

tfnet = TFNet(options)


def gen(camera):
    sess = tf.Session()

    with sess.as_default():

        while True:

            success, img = camera.read()

            if success:
                    results = tfnet.return_predict(img)


                    for result in results:
                        #tl = (result["topleft"]['x'], result['topleft']['y'])
                        #br = (result['bottomright']['x'], result['bottomright']['y'])
                        label = result["label"]
                        print(label)
                        cv2.rectangle(img,
                                    (result["topleft"]["x"], result["topleft"]["y"]),
                                    (result["bottomright"]["x"], result["bottomright"]["y"]),
                                    (255, 0, 0), 4)
                        text_x, text_y = result["topleft"]["x"] - 10, result["topleft"]["y"] - 10
                        cv2.putText(img, result["label"], (text_x, text_y), cv2.FONT_HERSHEY_SIMPLEX,
                                    0.8, (0, 255, 0), 2, cv2.LINE_AA)
                    cv2.imshow('frame',img)

                    ret, jpeg = cv2.imencode('.jpg', img)
                    frame = jpeg.tobytes()

                    yield (b'--frame\r\n'
                           b'Content-Type: image/jpeg\r\n\r\n' + frame + b'\r\n\r\n')
            else:
                print("Status of camera.read()\n", success, img, "\n=======================")

@app.route('/video_feed')
def video_feed():
    cam = cv2.VideoCapture(0)
    # cam.set(cv2.CAP_PROP_FRAME_WIDTH, 640)
    # cam.set(cv2.CAP_PROP_FRAME_HEIGHT, 360)
    #if cam.isOpened():
    #    print('opended')
    return Response(gen(cam),mimetype='multipart/x-mixed-replace; boundary=frame')


@app.route('/')
def webcam():
    return render_template('webcam.html')

if __name__ == '__main__':
    app.run(host='0.0.0.0', debug=True)

🚨오류: "AssertionError: Image is not a np.ndarray"

 => webcam 이 제대로 영상을 받아오지 못해서 발생함..

 => isOpened() == true 지만 계속 오류 발생

 => 재부팅 했는데 안고쳐짐

 => MAC ) 초록 불빛은 들어오는데 facetime도 안되고 다른 앱에서도 웹캠이 동작하지 않았다.

 => SMC 재설정 https://support.apple.com/ko-kr/HT201295

Mac의 SMC를 재설정하는 방법

 => 해결 😭😭

 

- Flask_darkflow 깃허브: https://github.com/yskim0/Handlang/tree/master/Web/Flask_darkflow

yskim0/Handlang

- 구현영상

  (목표)

   - 예측 결과 스택 쌓기

   - 와이어 프레임

   - 정확도 높이기