Welcome to sld.codes

On January 24th, my good friend Yannis Panagis and I entered the Developer Week 2022 Hackathon. Since last August, Yannis and I have been hacking together and always find hackathons a great way to challenge ourselves. In our day jobs, it's impossible to reach our full creative potential all of the time. As engineers, we can be slowed down by politics, design constraints, legal requirements, 100% test code coverage, logic that won't work, other teams, and the list goes on.

For me, hackathons provide a way to get out of those situations and flex those creative muscles. Remind yourself that when there's nothing in the way, you can be a creative machine and Developer Week 2022 seemed like an excellent opportunity to do just that. More specifically, the Symbl.ai + Agora challenge piqued our interest as we had never experimented with WebRTC before. The exact brief:

Combine Agora's rich interactive experience APIs with Symbl.ai's conversation intelligence (AI that understands natural human conversations) to build a more enhanced and smarter conversation experience for any use case of your choice.

The Idea

With every hack, Yannis and I try to keep it personal. We feel that if we are invested in the problem statement, the project will be more interesting to work on. So we asked ourselves - "Where in tech can we make conversations smarter?". One thing almost immediately came to mind - interviews.

As web developers, we've been through many tech interviews, and they can be frustrating - especially when those interviews don't even have someone else on the other side! We've all paid our dues with Hackerrank and Leetcode for the technical part of the process, but with little feedback and no practice alternatives, there was never a safe space to prepare for those recorded interviews once you make it past the tech tests.

Even once you finally feel ready to interview, you often don't receive any feedback at all to help you improve for the next one! The problem extends both ways - the harsh truth is that interviewing well is hard.

Our Solution

We built interview toast! A trifecta of three internal tools to help you prepare, get feedback and assess.

Interview Preparation Tool: There are a lot of great tools out there to help you prepare for the coding part of the application process. Interviews aren't just about code, and there aren't a lot of tools out there specifically designed to help you with the conversational part of the process. By leveraging Symbl's conversation intelligence AI in our video application, InterviewToast goes beyond simple speech-to-text and provides real-time insights and extracts keywords, sentiment and conversation analytics such as talk-to-listen ratios.
Interview Feedback Tool: By sourcing questions directly from the candidate they want to practice, InterviewToast extracts quantitative and qualitative feedback on practice interview sessions to help candidates get ready to find their next dream job.
Interview Assessment Tool: Our interview tool goes beyond the practice rooms straight into the interview battlefield. InterviewToast uses Agora to provide real-time transcriptions, keyword and profanity detection, and analysis on live interviews.

But the best way to explain it is by showing it! You can try it out at InterviewToast.com.

Fun fact!

The name "InterviewToast" has a double meaning! Toast can be undercooked or burned, just like an interview! But toast can also refer to the small alerts you get while using software applications.

How it was Built

elements={[ { id: "2", data: { label: (

<div>

</g>

</svg> Gatsby Functions

), }, position: { x: 350, y: 125 }, }, { type: "output", id: "2.1", data: { label: (

<div>

</svg> Real-time API

), }, position: { x: 0, y: 125 }, }, { type: "output", id: "2.2", data: { label: (

<div>

<g>

</g>

</svg> WebRTC Client

), }, position: { x: 175, y: 125 }, }, { id: "1.1", type: "input", // input node data: { label: (

</g>

</svg> Gatsby.js

), }, position: { x: 150, y: 0 }, }, { id: "1.2", type: "input", // input node data: { label: (

</svg> TensorFlowJS

), }, position: { x: 350, y: 0 }, }, { id: "3", type: "output", // output node data: { label: (

<div>

</g>

</svg> SendGrid Email

), }, position: { x: 250, y: 250 }, }, { id: "4", type: "output", // output node data: { label: (

</svg> Firebase

), }, position: { x: 450, y: 250 }, }, { id: "5", type: "output", // output node data: { label: (

<div>

</svg> Async API

), }, position: { x: 50, y: 250 }, }, // animated edge { id: "e1-2", source: "1.1", target: "2", animated: true }, { id: "e1-2", source: "1.1", target: "2.1", animated: true }, { id: "e1-2", source: "1.1", target: "2.2", animated: true }, { id: "e1-2", source: "1.2", target: "2", animated: true }, { id: "e2-3", source: "2", target: "3", animated: true }, { id: "e2-3", source: "2", target: "4", animated: true }, { id: "e2-3", source: "2", target: "5", animated: true }, ]} />

InterviewToast was built with:

Symbl.ai's for sentiment analysis, topic extraction, conversation analytics, entity and intent recognition, and real-time insights APIs.
Agora.io to enable vivid voice and video calling directly in the browser for real-time interviewer-candidate engagement
GatsbyJS as our blazing-fast React framework for performance, scalability, security and accessibility
Gatsby Cloud Functions to bring an entire backend to InterviewToast - without managing a backend
Firebase to enable user accounts and SSO.
TensorFlowJS MoveNet model for body keypoint detection to evaluate candidates posture and slouching in real-time for practice sessions

Challenges

WebRTC is something that scared me; I have been avoiding it until now. This hackathon gave me a chance to face that fear. While I am sure that our WebRTC integration is not perfect, I have learnt how to implement it and, now feeling a little more confident, I can't wait to work with it again.

Next Steps

Although this hackathon took place over two weeks, there were still a lot of features that we would have loved to implement that we did not have time for. The following is a list of some of the extensions we would like to implement next:

Stripe Integration: We can't monetise the site without integrating a payment API!
Pre-call Tests for Audio and Video: Verify whether the internet quality is good enough for the interview.
Better Stats: We love data visualisation! We would love to include interactive charts that drill down into the data (with D3!).

For more information on the product, read the Devpost here.