Home Assistant xAI Grok conversation agent

Home Assistant's year of the voice was an important milestone in a lot of ways. I recently purchased 3 of the Home Assistant Voice devices that have some reasonable specifications and I think are the missing piece to easily controlling Home Assistant with your voice.

In this guide I'm changing the conversation agent in Home Assistant to use the xAI Grok large language model to give my home the intelligence it needs to answer everyday questions while also being able to control the devices and automations setup in Home Assistant.

Some pre-requisites needed that we won't be going over today

• A working instance of Home Assistant
• HACS already installed
• Wyoming protocol configured with piper, whisper, and openWakeWord
• Voice Assistant setup in Home Assistant
• A microphone and speaker

Browse to https://console.x.ai/ and login easily with your x.com account. Create a team name to do the initial account setup. You can opt in to share your API data to get $150 credit.

!!!WARNING!!! This API does incur cost and any billing or charges are your responsibility and I do not take responsibility for any charges your may incur.

Add an API key by browsing to the key icon on the left menu

Click the Create API Key button

Enter a name like Home Assistant for the key and click Save

You'll need to copy the API key and proceed to the step 2 to install the custom integration and use this key to set it up

You can search for both xAI or Grok using the search box at the top of the HACS

Click on xAI Grok Conversation and then click Download

Click the Download button again to proceed with installation

Restart Home Assistant to finish the install

After coming back up browse to Settings -> Devices and services and click the Add Integration button in the bottom right corner.

When the search box comes up type in xAI or Grok to filter the results and click on xAI Grok Conversation

Enter the API key generated in the first step into the box prompting for the API Key and click submit

Choose an Area if desired but it is not required. Click Finish

Now the conversation agent has been setup so we need to edit our voice assistant to use the Grok conversation agent.

Browse to Settings -> Voice Assistants and click on your preferred Voice Assistant

On the voice assistant settings, click the conversation agent drop down box and choose Grok from the list and click Update to save the changes.

After saving your changes you should be able to choose the 3 vertical dots next to your Voice Assistant and choose 'Start Conversation' to chat with Grok to test some prompts.

At this point you should be able to trigger your voice assistant with your wake word and talk to Grok like its a home assistant.

Under Settings -> Devices and services find xAI Grok Converstaion and click configure

This will allow you to customize the initialization prompt that preps how Grok is going to respond to you. The default prompt works well to get short concise feedback.

Setup is complete and you should now have the ability to control your Home Assistant devices and ask questions from the all knowing Grok.