Turn an Old Phone Into a Smart Doorbell Camera With Facial Recognition

CREDIT

This how to would not be possible if it weren't for Juan Perez & Robin Cole creating this article on Hackster.io. Please check it out.

Announce Who Is Home Using Facial Recognition

ASSUMPTIONS

• You have a basic understanding of YAML
• You can run docker containers
  • Windows
  • Unraid
  • Raspberry Pi
• You have a Home Assistant instance running
• You have a Facebox instance running
• You have a MotionEye instance running

• IP Webcam - https://play.google.com/store/apps/details?id=com...
• MotionEye - https://github.com/ccrisan/motioneye/wiki
• Unraid - https://github.com/ccrisan/motioneye/wiki
• Docker - https://github.com/ccrisan/motioneye/wiki
• Facebox - https://github.com/ccrisan/motioneye/wiki
• Home Assistant - https://github.com/ccrisan/motioneye/wiki
• Discord - https://github.com/ccrisan/motioneye/wiki
• Google home devices

Add two input_booleans to your Home Assistance configuration.yaml file.

front_door_motion_sensor gives us a way to turn a switch off and on using MotionEye

front_door_motion_sensor_face_detect_this_pass We will use this later when we are making automations

configuration.yaml

input_boolean:
  front_door_motion_sensor:
    initial: off
  front_door_motion_sensor_face_detect_this_pass:
    initial: off

Below that we can add a binary sensor that ties directely to input boolean we just made.

front_door_motion_sensor gets it value directly from the input_boolean with the same name

binary_sensor:
  - platform: template
    sensors:
      front_door_motion_sensor:
        entity_id: input_boolean.front_door_motion_sensor
        device_class: motion
        value_template: '{{ is_state("input_boolean.front_door_motion_sensor", "on") }}'

1. In Home assistant click on the user profile name
2. Scroll all the way down
3. Click Create Token
4. Name the token
5. Save the token for later

1. Install IP Webcam from the app store
2. Start the app
3. Click start server
4. Save the URL provided from the app

*You will need to assign your phone a static IP so that the URL doesn't change

1. Click the hamburger on the left
2. Click the drop down
3. Click add camera...
4. choose Camera Type - Network Camera
5. Type the URL from the IP Webcam app and add "/video" to the end
   1. Example: http://192.168.1.121:8080/video
6. Click inside the camera input
   1. An MJPEG Network Camera should appear
7. Click OK

The camera from your phone should now be connected to MotionEye.

1. Scroll down to Video Streaming
2. Make sure Video Streaming is ON
3. Scroll down to Motion Detection
4. Make sure Motion Detection is turned ON
   1. You will have to adjust the settings in the area to make sure your MotionEye isn't detecting motion too much or too little
5. Scroll Down to Motion Notifications
6. Turn on Run A Command
7. Edited the following string and paste it into the Run A Command
   1. You will need your Home Assistant Long Lived Access Token that you made earlier
   2. You will need your Home Assistant IP address and port

curl -X POST -k -H "Authorization: Bearer [YOUR TOKEN GOES HERE]" -H "Content-Type: application/json" -d '{"entity_id": "input_boolean.front_door_motion_sensor"}'  http://[YOUR HOME ASSISTANT IP ADDRESS AND PORT]/api/services/homeassistant/turn_on

8. Turn on Run An End Command

9. Edited the following string and paste it into the Run A Command

curl -X POST -k -H "Authorization: Bearer [YOUR TOKEN GOES HERE]" -H "Content-Type: application/json" -d '{"entity_id": "input_boolean.front_door_motion_sensor"}'  http://[YOUR HOME ASSISTANT IP ADDRESS AND PORT]/api/services/homeassistant/turn_off

Now when Motion is detected you should see it reflected on the Home Assistant entity binary_sensor.front_door_motion_sensor.

Choose a file location to download a still image of the front door I chose '/surveillance/front_door/'

You will also need the URL to retrieve this still image. You can find this on your MotionEye web page --> Video Streaming --> Snapshot URL

configuration.yaml

shell_command:
  download_front_door_image: cd /surveillance/front_door/ && wget http://[YOUR MOTIONEYE IP]:[YOUR MOTIONEYE PORT]/picture/1/current/ && mv index.html front_door.jpg

Add two cameras to your Home Assistant's configuration.yaml file

configuration.yaml

camera:
  - platform: local_file
    name: front_door_local_file
    file_path: /surveillance/front_door/front_door.jpg
  - platform: generic
    name: front_door
    still_image_url: http://[YOUR MOTIONEYE IP]:[YOUR MOTIONEYE PORT]/picture/1/current/ ### MotionEye --> Video Streaming --> Snapshot URL

Add the Facebox image processing information to your Home Assistant's configuration.yaml file.

configuration.yaml

image_processing:
  - platform: facebox
    ip_address: [YOUR FACEBOX IP]
    port: [YOUR FACEBOX PORT]
    scan_interval: 10000
    source:
      - entity_id: camera.front_door
        name: facebox_front_door

The free version of Facebox lets you train 100 faces but does not retain what it learned after a reboot. To circumvent this issue we need to check on a regular bases that the faces are taught.

1. Download this python script - github
2. From your Home Assistant's appdata folder create a folder named facebox
3. Change directory into the facebox folder and create a folder named teach
4. Place the teach_facebox.py script into the teach folder
5. Open the teach_facebox.py scipt in an editor
   1. Change line 9 to match the IP of your Facebox instance
   2. Change line 10 to match the port of your Facebox instance
6. In the teach folder create folders that are named after the person who's face you want to teach to Facebox
7. In the personnel folders place photos of those people

We need to add a shell command in Home Assistant so that it can automatically call it when faces are not taught. Add the teach_facebox command under the download_front_door_image command

configuration.yaml under shell_command:

  teach_facebox: cd /config/facebox/teach && python /config/facebox/teach/teach.py

We need to create a binary_sensor that checks to make sure a face is taught. This sensor will run every 5 minutes. To do this we are going to need a random face to test off of. In the teach folder I created another folder called xyz. I got a random face off of This Person Does Not Exist and put that picture in the xyz folder.

1. In the teach folder create another folder called xyz
2. Download a picture of a random face from https://thispersondoesnotexist.com/
3. Put that picture in the xyz folder

configuration.yaml under binary_sensor:

  - platform: command_line
    name: facebox_taught
    command: curl -X POST -F 'file=@/config/facebox/teach/xyz/xyz.png' http://[YOUR FACEBOX IP]:[YOUR FACEBOX PORT]/facebox/check |  tac | head -14 | grep -i "name" | awk 'FNR==1 {print $2}' | sed 's/"//g' | sed 's/,//g'
    device_class: connectivity
    scan_interval: 5
    payload_on: "yes"
    payload_off: "no"
    value_template: '{%- if value == "xyz" -%} yes {%- else -%} no {%- endif -%}' 

Now for our first automation. We need to teach Facebox if the xyz face is not recognized.

automations.yaml

- id: teach_facebox
  alias: 'teach facebox'
  initial_state: on
  trigger:
  - platform: time_pattern
    minutes: '/1'
  condition:
  - condition: state
    entity_id: binary_sensor.facebox_taught
    state: 'off'
  action:
  - delay: 00:00:05
  - service: shell_command.teach_facebox

Trigger - When motion is detected at the front door

Condition - No faces have been detected this pass

Action - Run a command to download an image of the front door and send the image to Facebox to scan

automations.yaml

- id: 'while_motion_at_front_door_scan_for_faces'
  alias: while motion at front door scan for faces
  trigger:
  - entity_id: binary_sensor.front_door_motion_sensor
    from: 'off'
    platform: state
    to: 'on'
  - platform: time_pattern
    seconds: '/1'
  condition:
  - condition: state
    entity_id: binary_sensor.front_door_motion_sensor
    state: 'on'
  - condition: state
    entity_id: input_boolean.front_door_motion_sensor_face_detect_this_pass
    state: 'off'
  action:
  - service: shell_command.download_front_door_image
  - entity_id: image_processing.facebox_front_door
    service: image_processing.scan

Trigger - When Facebox detects faces in an image of the front door

Condition - Faces have not been detected this pass

Action - Turn on the input boolean front_door_motion_sensor_face_detect_this_pass

automations.yaml

- id: 'faces_detected_at_front_door'
  alias: faces detected at front door
  trigger:
  - event_type: image_processing.detect_face
    platform: event
    event_data:
      entity_id: image_processing.facebox_front_door
  condition:
  - condition: state
    entity_id: input_boolean.front_door_motion_sensor_face_detect_this_pass
    state: 'off'
  action:
  - service: input_boolean.turn_on
    data:
      entity_id: input_boolean.front_door_motion_sensor_face_detect_this_pass

Trigger - When faces have been detected this pass

Condition - The TV is not currently casting anything

Action - Stream the front door camera to the TV then wait till motion is no longer detected and the front door and then turn off the stream

automations.yaml

- id: 'cast_front_door_to_living_room_tv_if_face_detected'
  alias: "cast front door to living room tv if face detected"
  trigger:
  - entity_id: input_boolean.front_door_motion_sensor_face_detect_this_pass
    from: 'off'
    platform: state
    to: 'on'
  condition:
    condition: state
    entity_id: media_player.living_room_tv
    state: 'off'
  action:
  - service: media_player.play_media
    data_template:
      entity_id:
      - media_player.living_room_tv
      media_content_id: http://[YOUR MOTIONEYE IP]:[YOUR CAMERA PORT]
      media_content_type: image/jpg
  - wait_template: '{{ is_state(''binary_sensor.front_door_motion_sensor'', ''off'') }}'
  - service: media_player.turn_off
    entity_id: media_player.living_room_tv

Trigger - When faces have been detected this pass

Condition - Nothing is currently playing on the dining room speaker

Action - Set the volume to 50% then announce the names of the people whom faces were detected at the front door

- id: 'cast_TTS_front_door_to_speakers_if_face_detected'
  alias: "cast TTS front door to speakers if face detected"
  trigger:
  - entity_id: input_boolean.front_door_motion_sensor_face_detect_this_pass
    from: 'off'
    platform: state
    to: 'on'
  condition:
    condition: or
    conditions:
    - condition: state
      entity_id: media_player.dining_room_speaker
      state: 'off'
    - condition: state
      entity_id: media_player.dining_room_speaker
      state: 'idle'
  action:
  - service: media_player.volume_set
    data:
      entity_id: 
      - media_player.dining_room_speaker
      volume_level: .5
  - service_template: tts.google_translate_say
    data_template:
      entity_id: >
        {% if not is_state('media_player.dining_room_speaker', 'playing') and not is_state('media_player.office_speaker', 'playing') %}
          media_player.dining_room_speaker, media_player.office_speaker
        {%  elif not is_state('media_player.dining_room_speaker', 'playing') %}
          media_player.dining_room_speaker
        {%  elif not is_state('media_player.office_speaker', 'playing') %}
          media_player.office_speaker
        {% endif %}
      message: 'At the front door:
      {% for face in states.image_processing.facebox_front_door.attributes.faces -%}
        {% if not face.name == None %}
          {{ face.name }}
          {% if not loop.last %}, {% endif -%}
        {% endif -%}
      {%- endfor %}
      {% if states.image_processing.facebox_front_door.attributes.total_faces > states.image_processing.facebox_front_door.attributes.total_matched_faces %}
        + {{ states.image_processing.facebox_front_door.attributes.total_faces - states.image_processing.facebox_front_door.attributes.total_matched_faces }} unrecognised person(s)
      {% endif -%}'

Make sure you've integrated your discord and home assistant. HOW TO

Trigger - When faces have been detected this pass

Action Send a discord notification with a photo of the people whom faces were detected at the front door

- id: 'front_door_discord_notification_if_face_detected'
  alias: "front door discord notification if face detected"
  trigger:
  - entity_id: input_boolean.front_door_motion_sensor_face_detect_this_pass
    from: 'off'
    platform: state
    to: 'on'
  action:
  - service_template: notify.discord
    data_template:
      data:
        images:
        - /surveillance/front_door/front_door.jpg
      message: '
      At the front door:
      {% for face in states.image_processing.facebox_front_door.attributes.faces -%}
        {% if not face.name == None %}
          {{ face.name }}
          {% if not loop.last %}, {% endif -%}
        {% endif -%}
      {%- endfor %}
      {% if states.image_processing.facebox_front_door.attributes.total_faces > states.image_processing.facebox_front_door.attributes.total_matched_faces %}
        + {{ states.image_processing.facebox_front_door.attributes.total_faces - states.image_processing.facebox_front_door.attributes.total_matched_faces }} unrecognised person(s)
      {% endif -%}'
      target:
      - '[YOUR DISCORD CHANNEL ID]'

This is a very important automation to wrap everything up. Without it you will have some problems with people being announced that aren't actually there.

Trigger - When motion at the front door has stopped

Action - Set the input boolean value for front_door_motion_sensor_face_detect_this_pass to off and reset the while_motion_at_front_door_scan_for_faces automation

- id: 'turn_off_front_door_face_detected_this_pass'
  alias: turn off front door face detected this pass
  trigger:
  - entity_id: binary_sensor.front_door_motion_sensor
    from: 'on'
    platform: state
    to: 'off'
  action:
  - service: input_boolean.turn_off
    data:
      entity_id: input_boolean.front_door_motion_sensor_face_detect_this_pass
  - service: automation.turn_off
    data:
      entity_id: automation.while_motion_at_front_door_scan_for_faces
  - service: automation.turn_on
    data:
      entity_id: automation.while_motion_at_front_door_scan_for_faces