How to Make a GUI Virtual Assistant

Millions of people around the world use virtual assistants like google assistant, Siri, Alexa, Bixby etc........

There a lot of tutorials out there in the internet which teach people how to make their own virtual assistants with just a few lines of code.

But most of them are virtual assistants that run in the console and if you ask me those are quiet boring.

So today I will show you how to make a virtual assistant with a Graphical User Interface (GUI).

A few Prerequisites are:

1] Basic knowledge of python syntax

2] A laptop/desktop which has a microphone and a speaker

Note: 1] I am also a beginner programmer who just started learning 6 months ago. So if you find anyway to improve my code please tell me.

2] This Virtual Assistant will NOT be A.I integrated. You just tell it to do something and it does it So let's get started

3] Please download the starter file attached below

If you want to see a demo watch the youtube link given below.

https://github.com/Ascool7776/A-GUI-Virtual-Assist...

https://www.youtube.com/watch?v=dOJIYST6kjY

First of all you need to install python. I am using Python 3.9, but Python 3.X or higher might do.

Then we have to install a few packages using pip.

The first one is pyttsx3. to install it just type

pip install pyttsx3 

in your command prompt. I will show how you can do that in windows.

Just click the search icon on the taskbar and type 'cmd' as shown above.

Then a few more packages are

pip install SpeechRecognition 
pip install wikipedia 
pip install pyjokes 
pip install playsound 

There are many more modules that we will use but those are already built in to python.

import speech_recognition as sr    #To convert speech into text
import pyttsx3                     #To convert text into speech
import datetime                    #To get the date and time
import wikipedia                   #To get information from wikipedia
import webbrowser                  #To open websites
import os                          #To open files
import time                        #To calculate time
import subprocess                  #To open files
from tkinter import *              #For the graphics
import pyjokes                     #For jokes
from playsound import playsound    #To play sounds
import keyboard                    #To add keyboard activation  

Note:

A lot of these modules have overlapping uses. So you can get rid of the ones you don't want, but I like it this way

As mentioned before we will be using pyttsx3 to do this

Before writing the function to do this we have to assign a few variables.

name_assistant = "Cortana" #The name of the assistant


engine = pyttsx3.init('sapi5')   #'sapi5' is the argument you have to use for windows, I am not sure what it is for Mac and Linux
voices = engine.getProperty('voices')  #To get the voice
engine.setProperty('voice', voices[1].id) #defines the gender of the voice. 


# Note: voices[1].id sets it to female and voices[0].id sets it to male
<br>
<br>

Later we will add a feature to change the assistant's name, but for now lets keep it like this.

Now let's write the speak() function

def speak(text):
    engine.say(text)
    print(name_assistant + " : "  +  text)
    engine.runAndWait()

It's pretty self explanatory

Now you can type any thing in the speak function and the assistant will tell it out loud. But remember to put it in inverted commas

Now we are going to add a feature where the assistant will wish you when you open the program.

Normally its as simple as speak("Hello"). But we're going to do it a bit differently.

The assistant is going to wish 'Hello Good Morning'. The problem with this is that it can't wish good morning in the evening. So we have to assign a function.

This is where we are going to use the datetime module. In this evening is defined as after 6:00 p.m.

And afternoon is defined as after 12:00 p.m

The function looks like this:

def wishMe():
    hour=datetime.datetime.now().hour
    if hour>=0 and hour<12:
        speak("Hello,Good Morning")
   
    elif hour>=12 and hour<18:  # This uses the 24 hour system so 18 is actually 6 p.m 
        speak("Hello,Good Afternoon")
 
    else:
        speak("Hello,Good Evening")<br>

This function is also very intuitive.

And remember you have to call the function or else nothing will work

wishMe()

This function is a bit tedious because we have to write a list containing all the months and all the ordinal dates. To save the trouble for you I will just paste them here.

month_names = ['Janameary', 'February', 'March', 'April', 'May', 'June', 'July', 'August', 'September', 'October', 'November', 'December']
ordinalnames = [ '1st', '2nd', '3rd', ' 4th', '5th', '6th', '7th', '8th', '9th', '10th', '11th', '12th', '13th', '14th', '15th', '16th', '17th', '18th', '19th', '20th', '21st', '22nd', '23rd','24rd', '25th', '26th', '27th', '28th', '29th', '30th', '31st'] <br>

The function contains a variable for months and a variable for days both are which are later converted into integer values to pick out dates and months from the lists above.

def date():
    now = datetime.datetime.now()
    my_date = datetime.datetime.today()

    month_name = now.month
    day_name = now.day
    month_names = ['January', 'February', 'March', 'April', 'May', 'June', 'July', 'August', 'September', 'October', 'November', 'December']
    ordinalnames = [ '1st', '2nd', '3rd', ' 4th', '5th', '6th', '7th', '8th', '9th', '10th', '11th', '12th', '13th', '14th', '15th', '16th', '17th', '18th', '19th', '20th', '21st', '22nd', '23rd','24rd', '25th', '26th', '27th', '28th', '29th', '30th', '31st'] 
    

    speak("Today is "+ month_names[month_name-1] +" " + ordinalnames[day_name-1] + '.')<br>

You might ask why there is a -1 when we start picking out elements from the list. That is because The numbering for python lists start from 0 not 1. So we have to account for the extra 0th term.

Now if you call this function the assistant will tell you the date.

date()

This is a rather useful feature which will make quick notes and reminders for you.

def note(text):
    date = datetime.datetime.now()
    file_name = str(date).replace(":", "-") + "-note.txt"
    with open(file_name, "w") as f:
        f.write(text)

    subprocess.Popen(["notepad.exe", file_name])

The file's name will have the date and time in it. It will be stored in the same directory as your file.

The code is also quite simple. This is the only function in which the subprocess module will be used

To test it out just type:

note("Remind me to buy groceries")

and it works like a charm.

To achieve this we are going to use the speech recognition module.

def get_audio(): 

    r = sr.Recognizer() 
    audio = '' 

    with sr.Microphone() as source: 

        print("Listening") 
        playsound("assistant_on.wav")
        audio = r.listen(source, phrase_time_limit = 3) 
        playsound("assistant_off.wav")
        print("Stop.") 
        
    try: 
        text = r.recognize_google(audio, language ='en-US') 
        print('You: ' + ': '+ text)
        return text


    except:

        return "None"<br>

I will attach the two audio files. To call this function you have to do as shown below

text = get_audio()
print(text)

And it works.

Now get rid of all the functions you have called except the wishMe function.(Don't delete the whole function)

Now this is the fun part where you tell your Assistant to do stuff for you

How to do this??

It's simple you have to define a variable to the get_audio function and then check whether a few words are there in the variable and if it is there the assistant will have to speak or do something.

I have put all this under a function. So that it is easier to access later.

def Process_audio():

    run = 1
    if __name__=='__main__':
        while run==1:

            app_string = ["open word", "open powerpoint", "open excel", "open zoom","open notepad",  "open chrome"]
            app_link = [r'\Microsoft Office Word 2007.lnk',r'\Microsoft Office PowerPoint 2007.lnk', r'\Microsoft Office Excel 2007.lnk', r'\Zoom.lnk', r'\Notepad.lnk', r'\Google Chrome.lnk' ]
            app_dest = r'C:\Users\shriraksha\AppData\Roaming\Microsoft\Windows\Start Menu\Programs' # Write the location of your file

            statement = get_audio().lower()
            results = ''
            run +=1

            if "hello" in statement or "hi" in statement:

              wishMe()               


            if "good bye" in statement or "ok bye" in statement or "stop" in statement:
                speak('Your personal assistant ' + name_assistant +' is shutting down, Good bye')
                screen.destroy()
                break

            if 'wikipedia' in statement:
              try:


                speak('Searching Wikipedia...')
                statement = statement.replace("wikipedia", "")
                results = wikipedia.summary(statement, sentences = 3)
                speak("According to Wikipedia")
                wikipedia_screen(results)
              except:
                speak("Error")


            if 'joke' in statement:
              speak(pyjokes.get_joke())    
     
            if 'open youtube' in statement:
                webbrowser.open_new_tab("https://www.youtube.com")
                speak("youtube is open now")
                time.sleep(5)


            if 'open google' in statement:
                    webbrowser.open_new_tab("https://www.google.com")
                    speak("Google chrome is open now")
                    time.sleep(5)


            if 'open gmail' in statement:
                    webbrowser.open_new_tab("mail.google.com")
                    speak("Google Mail open now")
                    time.sleep(5)

            if 'open netflix' in statement:
                    webbrowser.open_new_tab("netflix.com/browse") 
                    speak("Netflix open now")


            if 'open prime video' in statement:
                    webbrowser.open_new_tab("primevideo.com") 
                    speak("Amazon Prime Video open now")
                    time.sleep(5)

            if app_string[0] in statement:
                os.startfile(app_dest + app_link[0])

                speak("Microsoft office Word is opening now")

            if app_string[1] in statement:
                os.startfile(app_dest + app_link[1])
                speak("Microsoft office PowerPoint is opening now")

            if app_string[2] in statement:
                os.startfile(app_dest + app_link[2])
                speak("Microsoft office Excel is opening now")
        
            if app_string[3] in statement:

                os.startfile(app_dest + app_link[3])
                speak("Zoom is opening now")


            if app_string[4] in statement:
                os.startfile(app_dest + app_link[4])
                speak("Notepad is opening now")
        
            if app_string[5] in statement:
                os.startfile(app_dest + app_link[5])
                speak("Google chrome is opening now")
                       

            if 'news' in statement:
                news = webbrowser.open_new_tab("https://timesofindia.indiatimes.com/city/mangalore")
                speak('Here are some headlines from the Times of India, Happy reading')
                time.sleep(6)

            if 'cricket' in statement:
                news = webbrowser.open_new_tab("cricbuzz.com")
                speak('This is live news from cricbuzz')
                time.sleep(6)

            if 'corona' in statement:
                news = webbrowser.open_new_tab("https://www.worldometers.info/coronavirus/")
                speak('Here are the latest covid-19 numbers')
                time.sleep(6)

            if 'time' in statement:
                strTime=datetime.datetime.now().strftime("%H:%M:%S")
                speak(f"the time is {strTime}")

            if 'date' in statement:
                date()

            if 'who are you' in statement or 'what can you do' in statement:
                    speak('I am '+name_assistant+' your personal assistant. I am programmed to minor tasks like opening youtube, google chrome, gmail and search wikipedia etcetra') 


            if "who made you" in statement or "who created you" in statement or "who discovered you" in statement:
                speak("I was built by Abhhi  Sannayya")

            
            if 'make a note' in statement:
                statement = statement.replace("make a note", "")
                note(statement)


            if 'note this' in statement:    
                statement = statement.replace("note this", "")
                note(statement)         

            speak(results)

I will explain a few things and clear somethings up in the next step.

I had promised at the beginning of the tutorial that this would not be a boring console assistant. But one with a User interface. We are going to add that right now.

To do this we are going to use tkinter a built in GUI module for python

To understand this better please watch a few tkinter tutorials on youtube.

Write the following code

We will add stuff to the functions later

def change_name():
     pass

def change():
     pass

def info():
     pass
def main_screen():

      global screen
      screen = Tk()
      screen.title(name_assistant)
      screen.geometry("100x250")
      screen.iconbitmap('app_icon.ico')


      name_label = Label(text = name_assistant,width = 300, bg = "black", fg="white", font = ("Calibri", 13))
      name_label.pack()


      microphone_photo = PhotoImage(file = "assistant_logo.png")
      microphone_button = Button(image=microphone_photo, command = Process_audio)
      microphone_button.pack(pady=10)

      settings_photo = PhotoImage(file = "settings.png")
      settings_button = Button(image=settings_photo, command = change_name_window)
      settings_button.pack(pady=10)
       
      info_button = Button(text ="Info", command = info)
      info_button.pack(pady=10)

      screen.mainloop()
    

  

Now you should get a screen that looks like the one above.

You probably got a working GUI now and if you press the star button, you can ask your virtual assistant anything

This is a feature you won't find in most other virtual assistants. Each assistant has its own fixed name.

But i don't like that a whole lot.

So our virtual assistant has the capability to change it's name.

To do this we have to fill in those functions we defined earlier.

I won't explain much of this code but if you don't understand it please watch a few tkinter tutorials

def change_name():

  name_info = name.get()

  file=open("Assistant_name", "w")

  file.write(name_info)

  file.close()

  settings_screen.destroy()

  screen.destroy()


def change_name_window():
    
      global settings_screen
      global name


      settings_screen = Toplevel(screen)
      settings_screen.title("Settings")
      settings_screen.geometry("300x300")
      settings_screen.iconbitmap('app_icon.ico')

      
      name = StringVar()

      current_label = Label(settings_screen, text = "Current name: "+ name_assistant)
      current_label.pack()

      enter_label = Label(settings_screen, text = "Please enter your Virtual Assistant's name below") 
      enter_label.pack(pady=10)   
      

      Name_label = Label(settings_screen, text = "Name")
      Name_label.pack(pady=10)
     
      name_entry = Entry(settings_screen, textvariable = name)
      name_entry.pack()


      change_name_button = Button(settings_screen, text = "Ok", width = 10, height = 1, command = change_name)
      change_name_button.pack(pady=10)

For this to work you have to create an empty file called 'Assistant_name'. Again I recommend you to watch a few youtube tutorials

And now if you press the cogwheel a new window will pop up prompting you to change the Assistants name.

Right now the wikipedia feature might not work a whole lot right because I forgot to define the wikipedia function. This is a function that displays whatever the assistant told onto the screen. Here is the code.

def wikipedia_screen(text):


  wikipedia_screen = Toplevel(screen)
  wikipedia_screen.title(text)
  wikipedia_screen.iconbitmap('app_icon.ico')

  message = Message(wikipedia_screen, text= text)
  message.pack()

This is relatively simple you just have to fill in the info function we defined earlier

def info():

  info_screen = Toplevel(screen)
  info_screen.title("Info")
  info_screen.iconbitmap('app_icon.ico')

  creator_label = Label(info_screen,text = "Created by Abhhi Sannayya")
  creator_label.pack()

  Age_label = Label(info_screen, text= " at the age of 12")
  Age_label.pack()

  for_label = Label(info_screen, text = "For Makerspace")
  for_label.pack()

So now you have a completely working Virtual assistant. Ain't that cool

I want to add the feature in which if you press a button, the voice assistant will get activated. We do this using the keyboard module. I thought we can use the f4 function key to do the job

keyboard.add_hotkey("F4", Process_audio)

This way you don't have to press the GUI button.

I will give you the whole code here. I highly recommend you not to just copy and paste the code but understand what it is that you are typing.

import speech_recognition as sr    #To convert speech into text
import pyttsx3                     #To convert text into speech
import datetime                    #To get the date and time
import wikipedia                   #To get information from wikipedia
import webbrowser                  #To open websites
import os                          #To open files
import time                        #To calculate time
import subprocess                  #To open files
from tkinter import *              #For the graphics
import pyjokes                     #For some really bad jokes
from playsound import playsound    #To playsound
import keyboard                    #To get keyboard
  
name_file = open("Assistant_name", "r")
name_assistant = name_file.read()

engine = pyttsx3.init('sapi5')  
voices = engine.getProperty('voices')  
engine.setProperty('voice', voices[1].id)
    
def speak(text):
    engine.say(text)
    print(name_assistant + " : "  +  text)
    engine.runAndWait() 


def wishMe():


  hour=datetime.datetime.now().hour

  if hour >= 0 and hour < 12:

      speak("Hello,Good Morning")
 
  elif hour >= 12 and hour < 18:

      speak("Hello,Good Afternoon")

  else:

      speak("Hello,Good Evening")


def get_audio(): 

    r = sr.Recognizer() 
    audio = '' 

    with sr.Microphone() as source: 

        print("Listening") 
        playsound("assistant_on.wav")
        audio = r.listen(source, phrase_time_limit = 3) 
        playsound("assistant_off.wav")
        print("Stop.") 
        
    try: 
        text = r.recognize_google(audio, language ='en-US') 
        print('You: ' + ': ' + text)
        return text


    except:

        return "None"


def note(text):
    date = datetime.datetime.now()
    file_name = str(date).replace(":", "-") + "-note.txt"

    with open(file_name, "w") as f:
        f.write(text)

    subprocess.Popen(["notepad.exe", file_name])


def date():
    now = datetime.datetime.now()
    month_name = now.month
    day_name = now.day
    month_names = ['January', 'February', 'March', 'April', 'May', 'June', 'July', 'August', 'September', 'October', 'November', 'December']
    ordinalnames = [ '1st', '2nd', '3rd', ' 4th', '5th', '6th', '7th', '8th', '9th', '10th', '11th', '12th', '13th', '14th', '15th', '16th', '17th', '18th', '19th', '20th', '21st', '22nd', '23rd','24rd', '25th', '26th', '27th', '28th', '29th', '30th', '31st'] 

    speak("Today is "+ month_names[month_name-1] +" " + ordinalnames[day_name-1] + '.')

wishMe()


def Process_audio():

    run = 1
    if __name__=='__main__':
        while run==1:

            app_string = ["open word", "open powerpoint", "open excel", "open zoom","open notepad",  "open chrome"]
            app_link = [r'\Microsoft Office Word 2007.lnk',r'\Microsoft Office PowerPoint 2007.lnk', r'\Microsoft Office Excel 2007.lnk', r'\Zoom.lnk', r'\Notepad.lnk', r'\Google Chrome.lnk' ]
            app_dest = r'C:\Users\shriraksha\AppData\Roaming\Microsoft\Windows\Start Menu\Programs'

            statement = get_audio().lower()
            results = ''
            run +=1

            if "hello" in statement or "hi" in statement:

              wishMe()               


            if "good bye" in statement or "ok bye" in statement or "stop" in statement:
                speak('Your personal assistant ' + name_assistant +' is shutting down, Good bye')
                screen.destroy()
                break

            if 'wikipedia' in statement:
              try:


                speak('Searching Wikipedia...')
                statement = statement.replace("wikipedia", "")
                results = wikipedia.summary(statement, sentences = 3)
                speak("According to Wikipedia")
                wikipedia_screen(results)
              except:
                speak("Error")


            if 'joke' in statement:
              speak(pyjokes.get_joke())    
     
            if 'open youtube' in statement:
                webbrowser.open_new_tab("https://www.youtube.com")
                speak("youtube is open now")
                time.sleep(5)


            if 'open google' in statement:
                    webbrowser.open_new_tab("https://www.google.com")
                    speak("Google chrome is open now")
                    time.sleep(5)


            if 'open gmail' in statement:
                    webbrowser.open_new_tab("mail.google.com")
                    speak("Google Mail open now")
                    time.sleep(5)

            if 'open netflix' in statement:
                    webbrowser.open_new_tab("netflix.com/browse") 
                    speak("Netflix open now")


            if 'open prime video' in statement:
                    webbrowser.open_new_tab("primevideo.com") 
                    speak("Amazon Prime Video open now")
                    time.sleep(5)

            if app_string[0] in statement:
                os.startfile(app_dest + app_link[0])

                speak("Microsoft office Word is opening now")

            if app_string[1] in statement:
                os.startfile(app_dest + app_link[1])
                speak("Microsoft office PowerPoint is opening now")

            if app_string[2] in statement:
                os.startfile(app_dest + app_link[2])
                speak("Microsoft office Excel is opening now")
        
            if app_string[3] in statement:

                os.startfile(app_dest + app_link[3])
                speak("Zoom is opening now")


            if app_string[4] in statement:
                os.startfile(app_dest + app_link[4])
                speak("Notepad is opening now")
        
            if app_string[5] in statement:
                os.startfile(app_dest + app_link[5])
                speak("Google chrome is opening now")
                       

            if 'news' in statement:
                news = webbrowser.open_new_tab("https://timesofindia.indiatimes.com/city/mangalore")
                speak('Here are some headlines from the Times of India, Happy reading')
                time.sleep(6)

            if 'cricket' in statement:
                news = webbrowser.open_new_tab("cricbuzz.com")
                speak('This is live news from cricbuzz')
                time.sleep(6)

            if 'corona' in statement:
                news = webbrowser.open_new_tab("https://www.worldometers.info/coronavirus/")
                speak('Here are the latest covid-19 numbers')
                time.sleep(6)

            if 'time' in statement:
                strTime=datetime.datetime.now().strftime("%H:%M:%S")
                speak(f"the time is {strTime}")

            if 'date' in statement:
                date()

            if 'who are you' in statement or 'what can you do' in statement:
                    speak('I am '+name_assistant+' your personal assistant. I am programmed to minor tasks like opening youtube, google chrome, gmail and search wikipedia etcetra') 


            if "who made you" in statement or "who created you" in statement or "who discovered you" in statement:
                speak("I was built by Abhhi  Sannayya")

            
            if 'make a note' in statement:
                statement = statement.replace("make a note", "")
                note(statement)


            if 'note this' in statement:    
                statement = statement.replace("note this", "")
                note(statement)         

            speak(results)


def change_name():

  name_info = name.get()

  file=open("Assistant_name", "w")

  file.write(name_info)

  file.close()

  settings_screen.destroy()

  screen.destroy()


def change_name_window():
    
      global settings_screen
      global name


      settings_screen = Toplevel(screen)
      settings_screen.title("Settings")
      settings_screen.geometry("300x300")
      settings_screen.iconbitmap('app_icon.ico')

      
      name = StringVar()

      current_label = Label(settings_screen, text = "Current name: "+ name_assistant)
      current_label.pack()

      enter_label = Label(settings_screen, text = "Please enter your Virtual Assistant's name below") 
      enter_label.pack(pady=10)   
      

      Name_label = Label(settings_screen, text = "Name")
      Name_label.pack(pady=10)
     
      name_entry = Entry(settings_screen, textvariable = name)
      name_entry.pack()


      change_name_button = Button(settings_screen, text = "Ok", width = 10, height = 1, command = change_name)
      change_name_button.pack(pady=10)


def info():

  info_screen = Toplevel(screen)
  info_screen.title("Info")
  info_screen.iconbitmap('app_icon.ico')

  creator_label = Label(info_screen,text = "Created by Abhhi Sannayya")
  creator_label.pack()

  Age_label = Label(info_screen, text= " at the age of 12")
  Age_label.pack()

  for_label = Label(info_screen, text = "For Makerspace")
  for_label.pack()

keyboard.add_hotkey("F4", Process_audio)


def wikipedia_screen(text):


  wikipedia_screen = Toplevel(screen)
  wikipedia_screen.title(text)
  wikipedia_screen.iconbitmap('app_icon.ico')

  message = Message(wikipedia_screen, text= text)
  message.pack()



def main_screen():

      global screen
      screen = Tk()
      screen.title(name_assistant)
      screen.geometry("100x250")
      screen.iconbitmap('app_icon.ico')


      name_label = Label(text = name_assistant,width = 300, bg = "black", fg="white", font = ("Calibri", 13))
      name_label.pack()


      microphone_photo = PhotoImage(file = "assistant_logo.png")
      microphone_button = Button(image=microphone_photo, command = Process_audio)
      microphone_button.pack(pady=10)

      settings_photo = PhotoImage(file = "settings.png")
      settings_button = Button(image=settings_photo, command = change_name_window)
      settings_button.pack(pady=10)
       
      info_button = Button(text ="Info", command = info)
      info_button.pack(pady=10)

      screen.mainloop()


main_screen()

Thank you.