So you want to run a study?

We ran a study to measure if using the Talk Hiring mock interviewing tool improves interviewing skills, and we have proven that it does! Here’s a link to our study setup and results. Running this study was much harder than I thought it would be. Here are my learnings for others who might want to run their own study!
Why did we run a study?
Our product helps job seekers become better at interviewing. To make a claim like that, we wanted to back it up with data. First we thought, why not measure hiring outcomes? There are too many confounding variables to measure hiring outcomes from people who use or do not use our tool; plus, it would take much longer to prove or disprove. Then we thought, can we use our existing interview data to measure interviewing improvement? Users practice with over 100 different interview questions across different industries. It’s hard to compare interview quality across different questions, let alone across different industries. We decided to run a study so that we could control variables and prove the worth of our mock interviewing product. Recruiting Participants:
As the operator of a self-funded startup, I initially tried to run the study without paying participants. Talk Hiring works with workforce development organizations mostly in NYC, and I tried to recruit participants through those organizations. We were able to recruit a few people through those channels, but the study would have dragged on for months if we had continued down that path. I tried running free mock interviews with people, and using those interviews as data points in the study, but that was taking up so much of my time with lots of people canceling/rescheduling. I finally decided to recruit for the study via a $10 Craigslist posting and through my existing Talk Hiring user base.

Foundations of Deep Learning

First blog post on the x.ai blog about Activation Functions in Neural Networks. You can read it here on Medium or here on the x.ai blog.

Ruthless Prioritization

Prioritization is one of the most complicated, contentious, and crucial aspects of building a startup. There’s always an order of magnitude more tasks that your team can take on. Making sure that your team is working on the highest value work items is vital. HBR (Harvard Business Review) just released a phenomenal article on prioritization. I’ll highlight the key points here and analyze them by pulling from my startup experience.

Mailbadger: A Postmortem

At the start of junior year at Duke, I was working on a startup called MailBadger with my friend, Kirill Klimuk. Essentially, MailBadger was a two-fold enterprise software solution. It helped companies automatically follow-up with their clients, and it displayed these results in a simple, user-friendly interface organized by client or by reminder. MailBadger was going to solve the problem of requiring employees to remind themselves to follow-up with their clients. Instead of reminding themselves to remind others, employees would simply set a reminder with MailBadger for their clients, and MailBadger would send an email reminder at the specified date and time.

word2vec in JavaScript

One of my personal goals is to better understand deep learning. Word2vec, an algorithm developed by Google, is one of the simplest (but still not simple!) forms of deep learning. Deep Learning is a type of machine learning when there is at least one hidden layer. Word2vec is an algorithm that develops vectors for words based purely on the assumption that similar words are often used in similar ways. By running words through this algorithm, you can find a vector for a word, and then take the cosine between 2 vectors to find how similar the words are…so cool! There are 2 ways that word2vec can be implemented.

• CBOW (Continuous Bag of Words) - The algorithm is given a context (n words before and after the word), and optimizes for the middle word. This is the strategy that I used and implemented.
• Skip Gram - The algorithm is given the middle word and optimizes for the context

Alarm Clock iOS App

I share my bathroom with a roommate. Nobody likes to wake up and find your roommate in the shower. To solve this problem, I built an Alarm Clock app that shares my wake up time with my roommate. That way, I can know when my roommate is planning on waking up, and I can set my wake up time accordingly.

NSNotification Debt

Disclaimer: NSNotifications are not iOS push notifications

My First Business Ever: KoalaCab

During the summer after my freshman year at Duke, my friend and I started our first business. I built my first website. It was a taxi sharing and taxi booking website called KoalaCab (I know Uber now does this, but it was pre-Uber). We ended up booking 450 cabs during our first year of operation.

Scaling Property of Randomness

I really enjoyed the book Fooled by Randomness by Nassim Taleb, and I came upon many interesting statements about randomness in markets, the most interesting one being the scaling property of randomness.

In the stock market, the general direction that stocks move in are for a reason, but the second or millisecond deviations that stocks have can be mostly attributed to noise or randomness. As Taleb did in his book, let’s take a stock that has a 15% return with 10% volatility per year. This translates into a 93% chance that the stock will go up in that year, assuming that the noise can be approximated by a normal distribution.

At a second time interval, there is only a 50.02% chance that the stock will go up. At a day granularity, there is a 54% chance that the stock will go up. At a month granularity, there is a 67% chance that the stock will go up, and at a year granularity, there is a 93% chance that the stock will go up. When I first read this, I was shocked. Randomness does not scale linearly, and therefore when analyzing the markets at a small time scale, stock price variations are often meaningless. This means that at the smallest of time scales, markets are inefficient, a very interesting conclusion.

Path

Path is a mobile-only social network for your closest friends. It provides the basics of a social network (messaging, news feed, picture/video posting, location check-ins, status updates, and “friending” of people). But, because it is more intimate than a larger social network (ex. Facebook), it has become a much richer and detailed way to document my life.

Things I Love Path is known for its design. Path’s best design elements are the clock, which changes as the user scrolls, and the “+” button that allows the user to add a new moment. Moving clock hands as the user scrolls is a truly innovative UI element, and is something that I have not seen on any application. Pressing the “+” button beautifully releases five different buttons that the user can select.

My favorite features in Path are emoticons, and book/movie/music/tv show sharing. Path allows emoticon posting on moments. Unlike Facebook where users are only able to ‘like’ a post, posts (or moments) on Path can be smiled at, winked at, frowned at, surprised at, or loved. This allows for rich and quick responses to a friend’s moment, while adding a unique feature to social networking.

I really enjoy book/movie/music/tv show sharing. My friends have shared lots of great media through Path that I did not know about, and I have discovered new media this way. Path makes it easy to share what music is currently playing on a user’s smartphone, and has autocompletion for most music, movies, books, and tv shows. When friends share songs they are listening to, other friends are able to listen to a preview of the song in the app, and when friends share movies or books, Path displays a short description of the movie or book in the app. The intimacy of Path promotes more sharing than on Facebook, and has allowed me to better stay in touch with my closest friends.

Things I Don’t One feature I never use is the ability to share sleep information. I have shared sleep information with friends, and every time friends either comment on me sleeping too much or me sleeping too little. The predictability of the responses to my hours of sleep gets old, and I no longer record my sleep data. I have approximately 30 friends on Path, and none of them share sleep data either. Sharing how many hours you sleep every day seems to be oversharing, and is an unnecessary feature of the application, in my opinion.

A Way to Monetize Path Path is currently trying to monetize by selling virtual goods, either as a monthly/yearly subscription or on a per item basis. This is a great start, but I am skeptical that this tactic will make Path enough money. I believe that sponsored stickers would improve the user experience while making Path and advertisers happy.

There are limitless options for sticker packs, and these would make the app much more fun while bringing in more money for Path. Sponsored sticker packs would be free for users, and would add more sticker variety for users to post. I know that Path is trying to build a social network without ads, but I think that these ads would not be intrusive at all.