Harris Osserman

Save the Date

2026-05-19T00:00:00+00:00

Over the holidays, a close friend was telling me about her dating life. She’d gone on a string of dates over the last few months. A few she thought went really well, but the other person never asked her out again. She had spent months going out, getting her hopes up, and walking away with no clarity about what was actually happening.

It reminded me of my own dating years ago. The dates themselves weren’t the hard part. The hard part was how long the whole process took. Months of guessing. Months of replaying conversations through your own biases.

I started thinking about how lopsided the dating industry is. Hinge, Bumble, Tinder, the League. Basically every dollar in this space goes to the top of the funnel. Profile optimization, photo selection, bio crafting, matching algorithms. But once you’re actually sitting across from someone? You walk away with only what you happened to pick up on.

That gap was the idea. What if you could get an outside perspective on a date you actually went on, not a coach reading your texts, not your friends speculating, but feedback grounded in what actually happened?

I called it Save the Date. The pitch was a mutual reflection system: both people opt in, both get their own private AI insights afterward, and the raw recording gets deleted. Nobody is reviewing their date’s review. The product wasn’t trying to gamify dating or turn it into a sport. It was trying to give you the one thing you can’t get anywhere else, an outsider perspective on every date.

The innovation I was most excited about was the feedback format itself. Instead of a clinical “here’s a summary of your date” report card, the insights came back as a Reddit-style thread. After every date, twelve different AI personas, each with their own voice and lens, would weigh in on what they observed. One persona is a no-nonsense friend who tells it to you straight. Another is the over-analyzer who pulls on a specific quote and unpacks it. Another is the romantic who sees the spark you missed. Another is the cynic. You scroll a thread of reactions the way you’d scroll a r/datingoverthirty post, except every comment is about you. The framing made the feedback feel like a group chat of friends instead of a verdict from an algorithm, and almost everyone I showed it to lit up at it.

I built the thing with Claude Code over a few months. React Native app, Supabase for the backend, AWS Lambda for the audio processing, Claude on the AI side. The hardest design problem was the consent flow. Both parties have to agree, in the app, before recording starts. The audio recording and the transcript both get deleted after the insights are generated. I obsessed over the privacy story because I knew that would be the thing people pushed back on.

Then I started doing user interviews. About 30 of them. Some were friends and friends-of-friends. The rest I recruited on Prolific, people who had actually been on multiple dates in the last year. I’d share my screen, walk them through the marketing site, the app, the consent flow, the post-date insights, and just let them talk.

What I expected was that people would have a privacy reaction. They did. But it wasn’t the privacy concern that mattered. The thing that came up in nearly every conversation, almost word for word, was:

“I like it. But how do you ask the other person?”

It came up from Joseph, a nurse in Boston who said his big worry was “getting the other person to be ok with it.” It came up from Emily in Portland, who said proposing recording “just feels awkward.” It came up from Evan in LA, who told me point-blank that he’d love to be the recipient but would never be the asker. It came up from Alyssa in Austin, who would happily use it but didn’t want her date to know.

Person after person told me the same thing. The product was interesting. The insights were exciting. They wanted to see what an AI thought of their date. But the act of sending a stranger a link and asking them to consent to being recorded? That was a wall.

So we iterated. We rewrote the marketing site to lead with trust and privacy. We added a “deletion receipt” that showed the user when their audio was destroyed. We added scripts, actual sample text messages, for how to ask your date, in three different tones (warm, casual, playful). We added a section on the site for how other people had handled the consent ask. We softened the colors. We added an AI personas page so people understood the feedback wasn’t coming from real strangers. We tried, in every way we could, to lower the awkwardness floor of asking.

The biggest unlock came from listening to the interviewees. The same people who flinched at the idea of asking a stranger to be recorded would tell me, almost as a throwaway, “I’d be happy to use it on a second date.” Joseph in Boston pitched me the line basically verbatim: “I had a great time, let’s do it again soon. By the way, I would love to try this app on our next date. No hard feelings if not.” That reframing changed the product. The consent ask doesn’t belong before a first date when you barely know each other. It belongs at the end of a date that went well, when the goodwill is highest and the next date is the natural conversation. We rewrote the scripts, the marketing site, and the in-app prompts around that timing. The awkwardness floor dropped noticeably.

But solving the consent ask still left a harder problem behind it, and it was the one I noticed late.

Even when someone downloaded the app, there was no natural moment to remind them to use it. Most people don’t have a date scheduled the day they hear about an app. They might have one in two weeks, or three, or not for a month. By the time the date came around, they had completely forgotten that Save the Date existed. I could send emails, push notifications, “any dates this week?” nags, but none of it felt good, and none of it solved the problem. The product needed to insert itself at exactly the right moment, and I had no way to know when that moment was.

The honest answer is that this product belongs inside a dating app, not outside one. The match itself is the right trigger. When two people match, the app already knows they’re talking. It can read the conversation and notice the moment they start discussing meeting up. It can offer Save the Date as a one-tap double opt-in, before either of them has to type a single awkward sentence to a stranger. The friction disappears. The reminder problem disappears. Both people opted in to a dating app, so they’re already on board with the premise of putting effort into dating. The consent ask becomes a feature, not a hurdle.

I’m not a dating app. I don’t have those matches. And building a dating app to get to this feature is the long way around.

Save the Date, the only STD you will want to tell your friends about.

Book Club for One Side Project

2025-05-26T00:00:00+00:00

I love to read, but I am a terrible reader. What I mean by that is my reading comprehension is frustratingly mediocre. I’ll zone out mid-sentence, wonder “wait, who is this character again?” and months later, struggle to recall even the basic plot. Back in high school, I used to force myself to add a sticky note to every page or two with brief summaries of what I’d just read, ensuring I was paying attention and retaining the story. It worked surprisingly well, but the downside was that it made reading painstakingly slow and significantly less enjoyable.

My dad once explained studying to me like this: your brain fills with information like a bathtub fills with water. After the test, the stopper gets pulled, and you’re left with just droplets clinging to the sides. Those droplets are the bits of knowledge that stay with you long-term. It’s a vivid analogy—and painfully accurate.

If I’m investing many hours into a book, I certainly don’t want to end up with just a little bathtub ring of knowledge clinging to the edges. I want to learn deeply, be genuinely moved, inspired to introspect, or pushed to see the world differently. This is exactly why people join book clubs—they crave that deeper connection to what they’ve read.

But what if you could have your own “personal trainer” for reading—a book club designed just for you? Imagine an automated, asynchronous buddy, invisibly guiding you to comprehend, reflect deeply, remember clearly, and suggest your next read. I spent the last couple of months hacking away at this idea purely for fun, without any pressure about the business side. That made this project uniquely joyful.

I began by exploring OpenAI’s capabilities to build a GPT model that would take the name of the book and, for a given reading session, the pages you started and finished at as input. The model would then provide a concise summary, highlight deeper themes, and suggest potential follow-up questions for further inquiry—questions you could ask the model or explore on your own. This works because many books we read are fully available online, whether legally or not. The results were impressive! Check it out yourself here.

I knew this would not be a final product solution, but more of an experiment of OpenAI’s capabilities. GPT or text-based interactions are awkward solutions. Interrupting your reading to open a phone or computer and manually input pages or chapters breaks immersion and inevitably leads to users forgetting to log their progress.

Now that the backend was working well enough to prove that it could work, I started a multi-week design sprint, trying to figure out the right way to capture the start and end page of a reading session without it feeling clunky or annoying. I spent weeks prototyping and playing with ideas. Could it be a watch app that lets you scroll quickly to log your spot? A voice command? A smart bookmark with just enough tech in it, like Mark? A tiny camera that clips onto the book or your shirt or glasses? Maybe something that senses page turns and just knows? I kept cycling through form factors, searching for something that felt seamless and subtle—like having a quiet little helper who always remembered where you stopped reading.

The brilliance of Granola, the invisible note-taking app that’s exploded in popularity, inspired me—it’s effortlessly unobtrusive, silently providing searchable notes and perfect recall of meetings without participant input. That “set-it-and-forget-it” elegance was exactly what I was chasing.

Eventually, cameras emerged as the most compelling solution. But if cameras were the answer, where should they be mounted? On a clippable book light? Attached via fish-eye lenses poking out from the book edges? Embedded in smart glasses or a wearable device clipped onto clothing?

After many brainstorming sessions with ChatGPT on form factor, I bought some Raspberry Pi gear to practically test different camera setups. Once everything arrived, it quickly became clear: glasses were undeniably the superior form factor. For a camera to accurately read an entire page, it must be a good distance away (about a foot). Placing cameras at the page edge wouldn’t cut it, requiring cumbersome setups with dual cameras awkwardly jutting out—and even then, they wouldn’t consistently capture the full baseline or header needed to reliably identify the page number.

This week’s Google IO introduced the Project Moohan smart glasses, Google’s answer to Meta’s Ray Bans. These seem exactly what my idea needs. Though their release is uncertain, likely around 2026, I’ll eagerly track their SDK developments. This whole idea really hinges on a product like smart glasses with a powerful SDK—something that lets you build tightly integrated, low-friction tools on top of always-on hardware.

In the meantime, Amazon, could you please integrate this feature into Kindle? You already have the perfect dataset on reading start/stop points, making this product effortlessly seamless. I’d gladly build it if you released an API…

Until the next rabbit hole.

Selling My First Company

2024-08-21T00:00:00+00:00

I did an interview with the Founder of Acquire.com on my journey as an entrepreneur, and the process of selling my first business, Talk Hiring.

Link here

Training ML Models

2024-08-13T00:00:00+00:00

Training Our Own Models at Talk Hiring

At Talk Hiring, we developed NLP-based text classification models to tackle two significant challenges:

Analyzing mock job interview transcripts: We focused on detecting the PAR method (Problem-Action-Result) in behavioral interviews and identifying rambling (e.g., discussing multiple unrelated examples).
Classifying job application emails: Using our Gmail-API integration, we built a model to classify emails based on whether they pertained to a job application and, if so, determine the status of the application (e.g., Interview, Offer, Rejection, Assessment).

In this post, I’ll outline the process we followed to build one of these models—our custom job relevance email classification model—highlighting what worked and what didn’t.

Overall Strategy

Why Build a Custom Model?

Given the availability and accuracy of models from OpenAI and Anthropic, why did we choose to train a custom model? The answer lies in scalability and cost-efficiency.

Our email parsers needed to run inference predictions on approximately 650,000 emails per day—a number that continues to grow. Processing thousands or tens of thousands of tokens per email, coupled with bursty email traffic, would have led to significant expenses and potential rate limiting from third-party providers like OpenAI or Anthropic.

The Power of Transfer Learning

Transfer learning became our preferred strategy. It allows us to fine-tune a pre-trained model, which has learned from large, generalized datasets, to meet our specific needs with a relatively small labeled dataset.

For this, we leveraged Wikitext-103, a widely-used base model for NLP tasks. Wikitext-103 is trained on about 100 million tokens from high-quality Wikipedia articles, providing a robust foundation of written English.

The Challenge of Email Classification

Training a model to determine whether an email is about a job application is more challenging than it might seem. Job seekers, particularly those who don’t unsubscribe from email lists, often receive a barrage of emails daily—both from real and automated recruiters. These emails often encourage the recipient to apply for jobs (e.g., “You’re a great fit for XYZ job. Apply today!”). Moreover, distinguishing between emails related to job applications and those related to other applications (e.g., loans, housing, school, or training programs) can be tricky.

Building the Training Dataset

Our dataset comprised about 750,000 emails between job seekers and employers, accumulated over time. For this specific model, we needed about ten thousand labeled emails. However, the dataset was highly unbalanced, as most emails people receive aren’t about job applications they’ve submitted. Ideally, we would label more emails, and can do that in the future to nudge accuracy up.

To avoid biasing the model towards predicting all emails as irrelevant to job applications, we constructed a training/test/verification dataset with a 50/50 split between emails about job applications and those that weren’t.

But achieving this split wasn’t enough. We also manually reviewed the dataset to ensure it wasn’t over-indexed on certain types of emails. For example, emails generated by Indeed for job submissions are automated and often look very similar, varying only by employer or job title. We didn’t want the model to excel at predicting machine-generated emails without also being adept at handling human-generated, job-related emails. To address this, I personally reviewed the training dataset to ensure that no more than a handful of examples of the same kind of email were included in our ~10,000 email training dataset.

Labeling the Data

We explored various options for labeling the data:

Scale.ai: While Scale.ai offered a distributed labeling process, the accuracy wasn’t high enough. We had to keep adding training examples and honeypot questions without achieving satisfactory results.
Synthetic Data Providers (e.g., Gretel): These solutions produced data that wasn’t realistic or sufficiently email-like.
Toloka: Ultimately, we found success with Toloka, which offered an in-house team capable of labeling datasets for startups on a budget. We collaborated closely with them for a few weeks to produce a high-quality dataset for model training.

Model Training

I’m a big fan of FastAI and its native support for transfer learning. Additionally, Google Colab—especially with GPU or TPU-boost—is excellent for saving work and running heavy ML workloads in the cloud. Here’s the process we followed:

Fine-Tuning a Language Model: We started with Wikitext-103 as the base model and trained it to predict the next word given the prior tokens. This language model training is unsupervised, allowing us to train it on many more email records than we have labeled. Once trained, we exported and saved the model weights for use in the next steps.
Training the Email Classifier: This step involves both art and science. We loaded the pre-trained language model and used a Long Short-Term Memory (LSTM) architecture to iterate through model training. Although attention-based architectures are more popular and accurate today, LSTM was our choice as this was a few years back. We started with a higher learning rate, gradually reducing it with each training iteration. The key was to plot learning rate vs. loss and choose a learning rate where the slope was most negative, indicating the most significant reduction in error rates. With each iteration, the learning rate was decreased to improve accuracy incrementally without overshooting. We were able to train a model with > 98% accuracy at predicting whether an email was job relevant or not.
Model Deployment: After training, we exported a .pkl file containing the model weights and stored it in a versioned AWS S3 bucket. This way, we could always revert to an earlier model version if needed. We then built a small Flask server running in Paperspace, equipped with load-balanced GPUs. This server dynamically pulls the appropriate .pkl file within a Docker container at build time to run inferences quickly and accurately.

Final Thoughts

I wish I could share my Jupyter notebook, but it contains sensitive email data. However, if you have any questions, feel free to reach out—I’m happy to share some code snippets or dive deeper into any part of the process.

Building Virtuous Product Loops

2024-07-31T00:00:00+00:00

All great consumer products need positive feedback loops, where events that happen via the product reinforce further usage of the product. Here’s how we built this kind of a virtuous, positive feedback loop within Hire Rewards.

The Challenge

Hire Rewards was struggling. We had a promise that users loved: earn cash back while you job search. However, skepticism was high. Many people feared it was a scam, a fear heightened by the fact that our users were mostly low/moderate-income Americans who are often targeted by scams. Many users shared vivid stories of being scammed, reinforcing their wariness.

To succeed, we had to perfect the core dynamic: earning rewards as you applied for jobs. There was no room for doubt.

The Core Dynamic

It was easy to explain to users that applying to a job would earn them money. However, the key was ensuring they received their rewards immediately upon application. Instant gratification was essential for a product like this.

Initial Attempts

Initially, we validated job applications via a Gmail integration. The problem was timing. We couldn’t predict how quickly after hitting submit the potential employer would send the automated “thank you for your application” email. Our educated guesses weren’t enough; telling users when they might get paid would lead to failure.

Next, we tried using webhooks—requests that our server would receive when someone applied for a job. These worked well but weren’t universally supported by all employers, leaving us with only a partial solution.

The Breakthrough

Our breakthrough came from an unexpected source: Ibotta, a popular grocery cash-back app. Ibotta requires users to take pictures of their receipts to ensure they bought the products they expect to earn cash back for. This receipt mechanism is used when Ibotta lacks a direct relationship with the grocery store’s cash registers.

With advancements in optical character recognition and text parsing from images, we realized we could employ a similar process. After users applied for a job, they could take a screenshot of the “thank you for applying” webpage, return to Hire Rewards, and upload the screenshot. We used a combination of automation and anti-fraud systems to verify that the screenshot was a legitimate job submission*.

The Positive Feedback Loop

This new screenshot process solved our core problem and created a virtuous loop. The user would go from our app to the employer’s job application webpage, fill it out, and then return to our app to earn their cash back. Once they uploaded a successful screenshot, the money was added to their account immediately.

After uploading a successful screenshot, the user was back in our app, with extra cash back in their balance, validated that the app was legitimate, and ready to jump into another job application. The average user applied to 9.45 jobs, far above the job board industry average of about 1 job submission per user.

Conclusion

By creating a seamless and immediate reward system, we built a positive feedback loop that not only reinforced the legitimacy of Hire Rewards but also encouraged repeated usage. This approach turned skeptical users into active participants, significantly boosting our engagement, job search outcomes, and profitability.

If you’re interested in learning more about how we built our automation and anti-fraud systems, feel free to reach out!

Earned Secrets

2024-07-31T00:00:00+00:00

For the last six years, I have been building Talk Hiring. Our goal has always been to improve the efficiency of the job training program (a.k.a workforce development program), job seeker, and employer triangle.

We started with automated mock interviews to enhance the efficiency of workforce development programs. Initially, we didn’t set out to build an automated mock interviewing tool, but we pivoted to that, and customers liked it a lot (we conduct ~35k automated mock interviews per year).

In the summer of 2021, we decided to reset and try something new to solve this problem.

I connected with five large employers and 15 well-respected job training programs in NYC. Workforce development programs hire a role called a job developer, who builds relationships with local employers and helps place job seekers into relevant jobs. To understand the challenges within this triangle of actors, I worked as a free job developer for these employers and workforce development programs.

Every week, I followed up with our employer partners and job seekers who had applied for these employer partners’ jobs to gather intel on how candidates were progressing in their hiring processes. All this information was relayed to the workforce development programs.

However, this process revealed several key challenges:

Lack of Incentive for Status Updates: Giving status updates to a workforce development program is not the job of either the employer or the job seeker. While it would be helpful, there is very little incentive for them to consistently share status updates.
Discrepancies in Progress Reporting: Employers and job applicants often (about 1 in 3 times) disagreed on where the applicant was in the hiring process. Employers were more often correct about the candidate’s status.
Inconsistent Response Rates: Both employers and job seekers were inconsistent in their response rates to status update requests, making it tough to pin down either party for updates.
Communication Gaps: Employers would email job seekers, but job seekers frequently missed these emails. This issue stemmed from ATS emails ending up in spam and job seekers being overloaded with emails.

Ultimately, having spotty job search status data for a workforce development program is like running a sales organization with spotty CRM sales data: it just won’t work that well.

I kept searching for a better way to track this data and realized that the job seeker’s email inbox was an underutilized system-of-record for identifying job search status. Almost the entire job search process sits in the job seeker’s inbox, from applications to job offers and everything in between. Sure, it’s a beast to structure this very unstructured data, but if we could do it, it would be very valuable.

We built a unique system to integrate into the job seeker’s inbox via a Gmail API integration. We parsed about 700k emails a day (~20 million emails per month) to identify job application-related emails. From there, we extracted information about the employer, job title, location, job status, and threaded the email into the relevant opportunity. We trained our own ML models using transfer learning and also leveraged OpenAI to achieve this. At our peak, we were parsing about 70k job application processes a month.

Job seekers are understandably hesitant to share their inbox with just anyone, so we integrated cash-back rewards. Job seekers could earn money for sharing this data, and we profited from the spread between what employers paid us to send traffic to their jobs and the cash back we paid to job seekers. We also shared hiring insights with job seekers who connected their inbox, sort of a “Glassdoor 2.0” where job seekers could see which jobs had the highest/lowest interview rates, how long it took to hear back about a job on average, and which jobs we thought were ghost jobs.

Leveraging Consumer Fintech Apps for Effective Distribution

2024-07-28T00:00:00+00:00

We were fortunate to discover a highly effective distribution channel for Hire Rewards, our cash back job searching platform: consumer fintech apps.

Consumer fintech apps make excellent partners for apps that help users earn or save money. These fintech apps often feature an “offer wall” presenting various opportunities for user engagement. By helping users save or earn money, these offers enhance user retention, increase bank balances (boosting revenue for the fintech app), and improve overall user satisfaction.

Cost Structures and Benefits

Fintech apps typically charge partners that want to be featured in their app on a CPC (cost-per-click) or CPA (cost-per-activation/signup) basis. Industry-standard CPCs range from $0.30 to $1. You can take your signup rate, and figure out what the CPA would be. While CPCs on Meta/Google/others can be similar or slightly higher, fintech apps offer a distinct advantage over other ad platforms.

Targeted Audience: Users on fintech apps actively seek out ads related to earning or saving money. They engage with your product with an open mind, unlike users on platforms like Instagram who often scroll past ads without interest.
Trust Factor: Users trust their financial institutions. If a fintech app that holds their money advertises your service, it is perceived as more trustworthy. This trust is not as prevalent with ads on social media or search engines. We saw that click -> signup rates via fintech apps were much higher than on other forms of digital advertising.

Seamless Integration

As a cash back app, we also benefited from the ability to closely collaborate with fintech apps to transfer money directly from our app into the users’ fintech app wallets. This tight integration enhances the user experience and adds significant value for the fintech app.

Scalability and Fast Partnership

The largest fintech apps have millions or even tens of millions of users. While this does not match the scale of Meta or Google, it still allows you to add hundreds of thousands of users to your app affordably. Fintech partners are typically quick to form partnerships and are financially motivated. If the partnership drives real revenue, they are eager to deepen or expand the collaboration.

There are at least 10-20 large fintech apps worth pursuing, making this channel a lucrative opportunity for distribution.

Partnering with consumer fintech apps proved to be a game-changer for Hire Rewards. By leveraging their user base, trust factor, and willingness to integrate closely, we were able to effectively distribute our product and achieve significant growth. For any app aimed at helping users earn or save money, fintech apps are a distribution channel well worth exploring.

How to Hire on Upwork

2024-07-27T00:00:00+00:00

Upwork’s wealth of high-quality talent, detailed bios, and ratings is unmatched. Over the years, I’ve hired at least 25 people through Upwork, and it has significantly contributed to my success. Here are some of the key strategies I’ve developed for hiring effectively on Upwork:

1. Avoid Paying for Boosts

I’ve found that paying for boosts is unnecessary. High-quality candidates consistently apply without the need for extra promotion. Save your budget and focus on evaluating the natural influx of talent.

2. Wait 24 Hours Before Reviewing Applicants

Patience is crucial. By waiting 24 hours after posting a job, you can get a clearer picture of the range of talent available. This waiting period allows you to identify both the standout candidates and those who might not be the best fit.

3. Disregard Upwork’s Recommended Salary Ranges

Don’t be constrained by Upwork’s suggested salary ranges. You can often find excellent talent at more reasonable rates.

4. Avoid Hiring Through Agencies

Hiring through agencies increases costs and introduces unnecessary middlemen. Directly engaging with freelancers ensures clearer communication and a more personal connection with your hires.

5. Conduct Quick Phone Screens for Large Applicant Pools

When faced with 50+ applicants, quick phone screens are essential. I use Calendly to set up 15-minute slots and invite about 30 candidates to interview over the period of 48 hours. This process (brutal for the interviewer) quickly filters out those who may look good on paper but fall short in areas such as communication skills, internet quality, and authenticity of their experience.

6. Follow Up with In-Depth Interviews and Work Simulations

After the initial phone screens, I usually identify a few strong candidates. I invite 3-5 of them for longer interviews that include work simulations. This could involve pair programming, writing customer emails, or designing components together. It’s a practical way to assess their skills and compatibility with your workflow.

7. Hire Two Candidates as a Safety Net

To mitigate risks, I often hire two top candidates. This strategy acts as a hedge against unexpected issues with freelancers. Typically, at least one of the two will prove to be a successful hire.

8. Be Cautious with Part-Time Freelancers

Hiring part-time freelancers (20 hours or less per week) can be challenging. They often juggle multiple jobs, making it difficult for them to prioritize your work. Full-time commitments generally yield better results.

9. Embrace International Talent

International freelancers can be as competent as those in the US and often at a lower cost. The quality and affordability of global talent are impressive. Additionally, international contractors are usually highly motivated to work with US companies, bringing enthusiasm and dedication to their roles.

10. Require all communication to happen on Upwork

When I post a job on Upwork, some freelancers invariably try to figure out my email address and email me, find my phone number and call me, or even start live chatting me on my company website. I tell anyone who does this that all communication must happen on the Upwork website. That way, I can stay organized, reduce distractions, and manage the speed of the hiring process.

Signals That You Are Building Something People Want

2024-07-27T00:00:00+00:00

When building a new product, it often feels like you’re walking through a dense fog. You may be confident that what you’re creating is important and needed, but user metrics and feedback can frequently lead to frustration, fear, and a lack of confidence.

On my journey to creating a product with strong product-market fit (where over 2/3 of users would be very disappointed if it didn’t exist), I discovered some key indicators that provided confidence I was working on something meaningful:

1. Heavy Customer Support Load

An influx of customer support inquiries is a positive sign. Users reaching out about issues and a high engagement rate with customer support indicate that they care deeply about your product. We often found ourselves inundated with support requests, which was a clear marker of user investment.

2. Genuine Curiosity

Hearing a lot of “wait, what?” when you describe your product is encouraging. This genuine curiosity from others is a great sign that your product is intriguing and worth attention.

3. Early Partnerships with Reputable Organizations

Securing partnerships with large, reputable organizations early on is a significant confidence booster. We were able to integrate with major, well-respected apps because these organizations saw the value in our unique offering, even when the product was relatively immature.

4. The “Too Good to Be True” Reaction

When users tell you that your product seems too good to be true, it can be a hurdle to overcome, but it’s also a sign that you’re on to something. If many people think your product’s promise is almost unbelievable, it means you have a compelling value proposition.

5. Strong Power Users

Having a few power users who are highly engaged and profitable, even if most users are not fully activated, is another positive indicator. These power users can provide valuable insights and validation.

6. Regular Requests for Feature Improvements and Bug Fixes

Frequent user requests for improvements and bug reports indicate strong engagement and a desire for your product to succeed. While it can be overwhelming to consider how to address all these requests, or even to figure out how to implement some of them, such interactions demonstrate that users are genuinely invested in your product’s future. Their feedback is invaluable and reflects their commitment to seeing the product evolve and improve.

A Caveat: Modern User Expectations

The old startup mantra of “if people are using a broken product, then you are on to something” is not as applicable today as it was 10-20 years ago. The overwhelming majority of users are now less willing to tolerate UI/UX issues or serious bugs. This shift makes it more challenging to gauge whether you are on the right path without developing a high-quality product from the outset.

Rethinking Pilots

2024-07-27T00:00:00+00:00

Pilots can be a challenging proposition for software companies. While they might seem like a good way to attract new customers, they often come with significant downsides. Pilots are typically offered at a lower cost than standard plans, leading to lost revenue. Additionally, once the pilot period ends, you face the task of convincing the customer to continue using your service.

Instead, I’ve developed an alternative approach that has proven to be more effective: offering a flexible monthly plan with an attractive initial discount. Here’s how it works:

1. Monthly Plans with No Commitment

Rather than a traditional pilot, I offer customers a monthly subscription that they can cancel at any time. This eliminates the pressure and uncertainty that comes with the end of a pilot period.

2. Attractive Initial Discount

To incentivize customers to choose a plan larger than the cheapest option, we offer a 50% discount on their first three months, only allowed for plans more expensive than the cheapest plan. When this discount is applied, it actually makes the larger plan cheaper than our cheapest plan for the initial period. This encourages customers to fully utilize all features and integrate the tool more extensively during what would traditionally be the pilot phase.

3. Encouraging Extensive Use

By making the more comprehensive plans more financially attractive, customers are motivated to explore the full potential of our software and implement it on a larger scale from the outset. This wider adoption during the discount period often leads to better integration and reliance on the tool.

4. Seamless Transition to Paid Plans

We find that most customers remain on the more expensive plan even after the discount period ends. The larger rollout and deeper integration during the initial months make it easier for customers to continue with the higher-tier plan. The key advantage here is that the default path is for customers to remain subscribed, reducing the need for a hard sell to continue the service.

Conclusion

This approach shifts the focus from a risky pilot to a more secure and profitable customer onboarding process. By offering a flexible monthly plan with a compelling discount, we not only attract more customers but also encourage them to stay on higher-tier plans long-term. This strategy ensures a smoother customer experience and a more stable revenue stream for the company.

By rethinking the way we handle pilots, we create a win-win situation where customers feel less pressure and are more likely to stick around, benefiting both their operations and our bottom line.

Viral Loops in Hire Rewards

2024-07-27T00:00:00+00:00

At Talk Hiring, we set out to reinvent the job search experience. We developed an innovative product called Hire Rewards, where job seekers earned cash back as they applied for jobs. It was a unique offering, the only one of its kind on the internet. Similar to how Rakuten and Ibotta provide cash back to attract consumers to brands, we discovered that offering a small cash back incentive (ex. $1 per job submission) was more effective in attracting high-quality candidates than spending that dollar to boost job postings in search results.

To grow our job seeker base, we needed to enhance our activation rate and motivate job seekers to invite others to our platform. One of our biggest challenges was overcoming skepticism; many people thought the concept of earning money to apply for jobs was too good to be true and possibly a scam.

Improving User Activation

We significantly boosted new user activations by offering a $5 signup bonus. The key to this strategy was “locking up” the bonus: users could only cash out their bonus once they had applied for at least three jobs. This requirement encouraged users to engage with the site, and once they did, they quickly realized that Hire Rewards was legitimate and worth their time. Our research indicated that $5 was the minimum amount needed to motivate users to give us a serious try.

Leveraging Referrals for Growth

To further accelerate our growth, we introduced a referral program offering $5 per referred user who signed up and successfully cashed out. Additionally, we provided $10 for approved social media posts about Hire Rewards. Referrals were crucial for us, as they added credibility to our seemingly too-good-to-be-true proposition. When people heard about Hire Rewards from friends, it felt much more trustworthy and real.

Outcomes

These tactics allowed us to scale Hire Rewards, achieve a 5:1 LTV to CAC ratio, and secure over 40% of our signups through referrals.

How to Automate Your Customer Support

2024-07-27T00:00:00+00:00

If you run a business and want to automate most of your customer support for cheap/DIY, follow these steps:

1. Organize Your FAQs

Take your frequently asked questions (FAQs) and put them in a simple, easily editable location. For us, this means using a Google Sheet.

2. Create a Contact Us Form

Develop a straightforward web form for users to contact you.

3. Integrate with an LLM

Every time a user submits the web form, pull all of the FAQs into the context window of a large language model (LLM) and ask it to answer the user’s question. We use Anthropic’s Sonnet model and manage around 100 FAQs.

4. Automate Response Emails

Send the user an email with their question and the LLM-generated answer, provided the LLM’s confidence level meets or exceeds a set threshold (for us, it’s at least 90%).

5. Provide a Follow-Up Option

At the bottom of the email, inform the user that they can reply to this email if they need further assistance, and someone from the team will review their query.

6. Monitor and Update

Regularly monitor the emails sent by the LLM to ensure the responses are accurate. If the LLM provides incorrect answers, step in to respond to the customer personally and update the Google Sheet to improve future responses.

🤖

By following this approach, we have automated more than 90% of our customer support, costing only a few cents per inquiry. This method requires no expensive customer support software and involves minimal setup time.

Getting into Techstars

2021-02-12T00:00:00+00:00

Today was my last day of the innaugural Techstars Workforce Development Accelerator. It was the greatest entrepreneurial learning experience of my life. But, in this post, I’m not going to talk about that. I’m going to bring you back in time to September 17, 2020.

Over a period of 73 days from July 6 to September 17, I had 7 interviews with Techstars. I was also sending biweekly business status update emails to them. Final decisions for the class were due on September 18, and I still hadn’t heard by the morning of the 17th. I figured I was extremely borderline, and couldn’t sleep. I got out of bed early in the morning and typed out the following business update email, sending it at 6:33 AM.

Here is what I sent them:

Hey Taylor and Allie,

2 days until the final decision! I might as well make your decision a little easier by sending over a strong update:

Video mock interviews are shipped, live, and getting rave reviews. Still a few bugs that happen sporadically to users that I’m trying to identify and fix. I added TrackJs into our stack on Tuesday so that we can better identify and fix frontend bugs.

Anson and I are brainstorming/prioritizing the next few things to work on. We have converted exactly 1/3 of programs that have started a free trial (7 out of 21). When a program sees strong usage in the free trial, they always at least try to purchase Talk Hiring for their organization post-trial. Our goal should be to get programs to the point where they see the value (aka lots of mock interviews completed in trial), and would like to purchase; whether or not the budgets/security/privacy/timing is in our favor is something somewhat out of our control. We’re writing out specs for projects that would help programs get mock interviews going as easily as possible (ex. CSV roster upload + assigning mock interviews to students, dynamic onboarding checklists on our website).

Seeing more mock interview usage…fall is certainly upon us! Fall + spring are our busier seasons because more programs are in session. 161 mock interviews completed this week so far (we did 355 last month in total).

Anson is working on a website redesign. Our product has changed so much in the past year since the last website went live, and we think that our sales + product usage could really be improved through a better looking + more informationally up-to-date website. Plus, he’s a much better designer than me.

In the last 1.5 weeks, we have started free trials with –redacted–, –redacted–, and –redacted–. 2 organizations (–redacted— and –redacted–) are bought in to the value of Talk Hiring, but are working internally to get legal/security/privacy approval before purchasing. We’re partnering with Safebase to help us with security/privacy audits going forward (very important to schools/colleges).

Got verbal word last night that the –redacted– continues to love Talk Hiring and wants a renewal annual contract. Due to security/privacy reasons, they really like that we support audio-only.

Acquisition/distribution experiment updates:

I’m getting much better at automated cold emails. On 126 contacts that are in workforce development + part of this automated email sequence, I’ve booked 3 meetings so far. A 2.4% meeting booking rate on a cold, automated email is amazing for me.

Testing the waters for a feature/acquisition tool. The premise is that workforce development programs that don’t use Talk Hiring almost certainly do mock interviews with volunteers. These programs are often posting on social media to find those volunteers, so it’s extremely easy for me to know who runs volunteer-led mock interviews + could be a good user of a tool that supports volunteer-led mock interviews. A product that supports volunteer-led mock interviews does not exist in the market and could be a good way for programs to see what Talk Hiring’s automated feedback is like. I posted a link on about 30 LinkedIn posts asking for volunteer mock interviews and got 8 responses. The next step is to talk through how this product could work with those 8 programs and see if they’re still interested.

Survey feedback that we’ve received from users this week:

What is the main benefit that you get from using Talk Hiring?

“i get to prepare for my upcoming interview”

“Ability to feel less nervous”

“Learning how to take my time before answering the question”

“I get to practice and perfect my skill of interviewing”

“Preparing for an interview”

“Being prepared to answer anything you are asked”

“Able to have positive feedback on my answers.”

“Be more comfortable with interviews”

How can we improve Talk Hiring for you? We read every piece of feedback you share with us!

“I’m not so sure because I enjoyed the interview so much.”

“Love the Talk Hiring”

“I think you did very good and don’t have to improve anything.”

“The question where good as they where.”

“I think the program is pretty awesome already.”

Plus we published a new case study: https://www.talkhiring.com/success-stories/the-hope-program

Follow-up to your two questions, Taylor, the last time we talked on the phone:

How big of a company am I trying to build?

I’m averse to the model of “raise a bunch of money and then race against the burn rate to get profitable.” I’m certainly not averse to raising money (and plan to), but I don’t want to raise before I have seen that Talk Hiring is ramen profitable or near ramen profitable. Being financially strong will help us raise at a higher multiple/valuation while giving us a strong base from which to grow the business. That’s certainly something that every investor wants :)

You see a big opportunity for us to work with employers and wanted to know if we plan on doing that.

YES, that is 10000% the plan. Here’s how we plan to do it. We already know which workforce development program users are enrolled in and which industries they are looking to enter (based on the industry-specific question sets they choose). With that information, we have a good idea of where they live, what they want to do, when they will be job searching, and the quality of their instruction. In the mock interview automated feedback page, we can start surfacing something like: “company x in your area is interested in hiring for the role that you just mock interviewed for. Would you like to send this interview over to them and make this mock interview into a real interview?” We can start making the employer/job seeker connections this way, and once that is proven to work, we can build a formal job searching component and standalone voice-based job applications to apply to those jobs! The opportunity to build a tech-accessible + quick to fill-out job application that actually presents the candidate in a positive yet realistic light is enormous. I don’t need to convince you of that. I beta-tested this idea with a crappy splash page, Google Ads, and a 1-800 phone number back in the spring of 2018 and found over 40 users calling in to use our beta voice-based job application. Reviews of talking instead of writing to create job applications were extremely positive from the job seeker side in this beta test. If this works, it would help Talk Hiring grow within workforce programs; these programs are already spending lots of time helping candidates with job applications, resumes, cover letters, computer access, spelling, and grammar, and that time could be better spent in so many other ways. And then at that point, our mock interviewing tool becomes free/cheap + a user acquisition tool for our hiring marketplace.

At 4:16 PM that day, I got this email from Taylor:

Harris, when you have a moment, can you give me a call?

When I called, he started off with all the reasons why Talk Hiring is great. I was waiting for him to say “but we have decided to put you on the waitlist” or “but the process was extremely competitive, and we have declined you a spot in this year’s cohort.” Instead, I never got the “but” and am so grateful :)

Startup Ideas: Wouldn't it be cool if...

2020-08-13T00:00:00+00:00

I’ve been jotting down startup ideas as they come to me for years. Most, if not all of them, are terrible. I thought it would be fun to share some of them, in case they are helpful to someone brainstorming:

Website uptime insurance — We know that brief website outages can be extremely costly, especially for high-volume e-commerce sites. For example, Amazon had partial outages on Prime Day back in 2018 and it cost them upwards of $75 million. The challenge is that engineers can intentionally bring down websites, so you would have to have some sort of way to insure only certain kinds of outages. To get historical data, you could ping the website every x seconds to see what sort of uptime baseline each website has.
Contextual info for Google Calendar — It would be interesting if there was a tool that could give you helpful information about each one of your upcoming events. For restaurant reservations, you’ll get top-rated meals. For sales pitches, you’ll get content on how to sell. For virtual social gatherings, you’ll get some fun, zoom-friendly games to play. If the product is really good here, there’s an opportunity to sell ads as well.
Fruit/vegetable washing machine — It always seemed sort of dumb to me that you go to a grocery store, you pick your fruits/vegetables off the shelf, you put them in a bag, and then you have to wash them when you get home. Why not wash them before putting them in those produce bags? Grocery stores already sell pre-washed produce in plastic containers. What about a “washing machine for produce” that you could use at the end of your shop or before you put the fruit/veggies in those plastic bags? Maybe it wouldn’t be hygienic and it possibly would make the food spoil faster, but these are things to look into. Just a fun thought.
Alcohol-free bar — About 30% of Americans don’t drink alcohol. What if there was a bar catered to them? The drinks could be similar to what a smoothie bar serves, but with a bar atmosphere. The menu would have to be entirely non-alcoholic. If the smoothies are tasty enough, I think a lot of health-conscious alcohol-drinking people would enjoy the change of pace.
Tablet-based online preschool — 60% of 3-year-olds don’t attend preschool, usually due to cost. I wonder if someone could build a much cheaper preschool that was entirely virtual, live-sessions, and tablet-based. A hard, and maybe impossible challenge, but something worth trying. Studies have shown that increased screen time in toddlers is associated with poorer verbal skills. However, these studies are correlative rather than causative.
Tom’s model but for dress clothes — Why not make a buy-one give-one company for dress clothes? There are already nonprofits out there ready to distribute dress clothes to people in need of professional attire. Probably not the right time to start something like this with sales for dressy attire way down due to COVID.
Enterprise Security as a Service — When a software vendor is trying to sell its product/service to an enterprise, there is always a security and privacy audit of the vendor. There are standardized security and privacy certificates that vendors can get approved for (ex. SOC 1 & 2). But, wouldn’t it be great if there was a 3rd party that did a deep audit of a company’s security and privacy and shared that with prospects? This company could also answer the common security and privacy questions that will be asked by vendors. It would save tons of time on both sides!
Voice-assistant necklace — Some people don’t hear well. Wouldn’t it be cool if there was a device that was listening, and if the wearer pressed a button or said “what”, it would repeat what it just heard? Maybe it would just save the last minute of audio and delete everything else. Of course, there are lots of privacy concerns and usability questions, but I think there is be a market for something that can help people with hearing loss better participate in conversations.
Better use of utm_term — When a user clicks a Google ad, the destination website is sent a parameter in the URL called the utm_term. This field stores the search query that the user searched on Google. Wouldn’t it be cool if there was a tool that could dynamically change your website to reflect what the user searched in the utm_term?

How Talk Hiring Started Working With High Schools

2020-07-28T00:00:00+00:00

Back in February of 2019, we had just started piloting our mock interviewing tool with a handful of career readiness programs. We had very few users, and I personally knew every person that was rolling out Talk Hiring in their organization.

One day in late February, Talk Hiring had a spike in mock interviews. All of the users were coming from a single high school in NJ. This was exciting, but also surprising! Who was spreading the tool and how did they know about our little company that had just launched?

I emailed a few of the students, asking them for the name of the teacher that told them about Talk Hiring and for some quick product feedback (in exchange for an Amazon gift card). Not surprisingly, that Amazon Gift Card was enough incentive to lead me to the teacher who found us, Chrissy Russell.

I then reached out to Chrissy, and had a wonderful email exchange with her about Talk Hiring!

But it didn’t end there!

In November of 2019, we saw another spike in mock interviews, not only with Chrissy’s class, but also with other schools across the United States. I emailed Chrissy again, to see if she was the instigator behind all of this :)

One of the teachers that checked out Chrissy’s Facebook post was a teacher in Douglas County, GA. She tried the tool and liked it so much that we ended up working with the entire school district.

We owe an enormous amount of appreciation to Chrissy for taking a chance on us and for bringing Talk Hiring to high schools!

Customer Feedback as a Product Loop

2020-07-02T00:00:00+00:00

I recently discovered Kevin Kwok’s Blog. I especially enjoyed his posts about the product loops in Figma and Superhuman. I was talking to Matt Ayers this week, and he helped me realize that I’m building a product loop, but this one is around customer feedback.

At Talk Hiring, we’ve built a mock interviewing tool for career readiness programs. High schools, colleges, veteran programs, and more general workforce development organizations purchase Talk Hiring to improve the interviewing skills of job seekers. For Talk Hiring to be successful, it obviously has to be liked by both staff and job seeker end-users.

We collect all kinds of user feedback, but one of our primary forms of feedback comes through surveys. We send end-users an optional, 3-question survey after every mock interview. The interesting twist is that we share this feedback with the staff at the relevant career readiness program.

What are the loops here?

Staff at career readiness programs nudge end-users to fill out the surveys to get insight into whether Talk Hiring is worth paying for. Every Monday morning, we send an automated email with all the feedback received in the past week. We also display this feedback in real-time dashboards for staff. When feedback is positive, programs are more motivated to keep using the tool.
- And it’s important to note that when feedback is very negative, that can bring on a negative feedback loop. If your product is very early and getting lots of negative feedback, it’s probably not a good idea to share feedback this openly.
Every prospect wants to evaluate Talk Hiring qualitatively and quantitatively. By automatically tracking and reporting on qualitative benefits, as well as quantitative ones, we make it easier for new programs to evaluate if they want to pay for Talk Hiring. If we’re confident in how the metrics will turn out, which we are, we’re making it easier for prospects to move forward in the sales process. And as my mentor and coach always says, “business begets business.”
I get more user feedback to improve the product, which should result in more users, and then more user feedback.
I’m collecting feedback to quote in success stories, which I then use as sales collateral.

Free Trials: Do they see the benefit?

2020-06-20T00:00:00+00:00

“Do they see the benefit?” is what my business coach always asks me when we’re going down the list of where prospects are in the sales process. It’s all about structuring a sales process that helps the customer see that the benefits far outweigh the costs. Pricing less than customer value is a good thing! My customers are nonprofit job training programs and public schools, so I have an especially uphill battle.

We sell an automated mock interviewing tool called Talk Hiring to career readiness programs. When we were in our free pilot stage, we asked how prospective customers wanted to evaluate Talk Hiring, and three aspects kept popping up.

Are people using this tool?
Do they like using the tool?
Are they improving their interviewing skills?

Once I figured out that this is often how Talk Hiring was being evaluated, it made it really clear how we should structure our free trials. We need a free trial process that has the following characteristics:

Long enough for students, teachers, and administrators to really use Talk Hiring.
Short enough that prospects maintain their motivation to move along in the sales process.
Structured in a way that the above interviewing metrics can be very positive.

What we settled on

We offer usage-based free trials that incentivize usage and rapid rollout. In each free trial, an organization can do 50 mock interviews on Talk Hiring for free. But, if they use up their 50 mock interviews within the first 45 days of their free trial, they get another 50 for free (100 total). Free trials expire after 6 months.
Prospects are incentivized to roll out Talk Hiring quickly because of our 45-day cutoff. And if they’re not rolling it out, the 45-day cutoff is a great thing to remind prospects of via email. “If you do at least x mock interviews in the next y days, you’ll get another 50 mock interviews for free.”
In a standard usage-based trial, the metric of usage (in this case, mock interviews) would be viewed as a scarce resource. But in our case, we’re getting prospects in the mindset of viewing mock interviews as the opposite.
Having a time cutoff is important just so that free trials don’t linger forever. 6 months is long enough to give prospects a few different opportunities to try to roll out the tool, but not so long that free trials can linger forever.
Any and all staff at prospective organizations that want dashboard access can get dashboard access during free trials. Our dashboards are structured to track the real-time data to answer the 3 ways that most prospects evaluate our tool (ex. do they use the tool, do they like using it, are they improving their interviewing skills). Dashboard below:

To give a little more context, I felt like it would be helpful to give you some background on how I structure my sales process.

Initial contact. About 40% of my prospects come from referrals (thanks Apollo — https://www.apollo.io/), another 40% come from cold emails, and the other 20% come from events, social media, and other smaller sources.
Try to start a free trial as quickly as possible. Some programs want to talk with me first, others don’t want to. The only step required from them to start a free trial is to e-sign a free trial agreement (thanks Contractbook — https://contractbook.com/).
Once the agreement is signed, do an onboarding meeting to outline best practices for rollout, walk through the tool, talk through the goals of the trial, etc. Write up the notes in a shared Google Doc that can be referenced throughout the trial.
Track usage during the trial (thanks Metabase — https://www.metabase.com/), and give nudges along the way.
Once the trial is over, shut off their account and see if they want to purchase. The details of what happens in step five is definitely worthy of another blog post!
If they want to purchase, send over a paid contract agreement via Contractbook and an invoice via Stripe.

I'm not doing enough

2020-06-07T00:00:00+00:00

I was reading Clair Minson’s article on how workforce development needs a reckoning. I’ve felt many of the same feelings.

“When I started focusing on the intersection of race, racism and workforce development, I was forced to face the reality that I too had played a role in keeping Black and other non-Black workers of color economically oppressed. By designing training programs to “meet the needs” of employers rather than challenging the institutional policies and practices of employers, I was upholding the status quo. Every time I framed the jobseekers I worked with as “low-skilled” I was reinforcing the negative cultural messages that Black people and other non-Black people of color aren’t skilled and therefore need to be “taught how to be job ready”. And every time I chose to connect jobseekers to low-wage, dead end jobs, I sealed the fate of their entire family because of the harsh reality that career advancement opportunities for Black and non-Black workers of color in predominantly white industries are rarely accessible to them.”

I’ve been working in workforce development for 2+ years. Here are the stories of some of the people that I’ve worked with:

A young man was trying to make some money during the holiday season. He was commuting from his home in the Bronx to a UPS packing facility in Secaucus, NJ. His shift started in the middle of the night, after the last trains ran from NY to NJ and before the morning rush. He used to take the last train to NJ at 1 AM, and sit or sleep on a bench in the train station until his shift. He would bring his bike so he could bike from the Secaucus station to the UPS packing facility, all during the bitter cold of December.
I sat with a woman and helped her apply for a job that required a valid driver’s license. She had to renew her license, as hers had expired. I sat with her as we went through multiple security steps to recover her bank password. Once we had logged in, I saw that her bank account had a balance of only $12.
A man dreamed of opening up his own restaurant one day. In the meantime, he worked as a line cook at a restaurant in Hell’s Kitchen. He lived in the Bronx, and he had to leave his home 2 hours before his shift started, simply because the NYC subway system was so unpredictable.
A young man had to be sure that his job training wouldn’t interfere with his trips to the methadone clinic.
A woman was working in an office in the suburbs of Connecticut. She was robbed and physically abused when walking back to her car at the end of the workday. This trauma resulted in years in and out of homelessness. She told me this because she didn’t know how to describe this gap in her resume to employers.
Countless people could not write above an elementary school level, even though they had graduated high school or obtained their GED.

Workforce development is proven to help people get better jobs, higher pay, and more valued skills. But as a community, we are too often satisfied when someone goes from chronic unemployment to minimum wage, or from minimum wage to a low wage. Of course, these are steps in the right direction, but the predominantly black and brown people that we serve deserve better. I’ve been trying to figure out how best to support inequality-reducing initiatives these days.

Request for startup

We all know that social services are chronically underfunded. I was reading JVS’ Catapult Report over the weekend and learned about Social Impact Bonds, a blending of private and public capital. There are plenty of articles outlining the problems with social impact bonds, but service providers will tell you that there are plenty of challenges when getting paid by the government too. How these bonds work:

A public entity, usually a state or local government, issues debt.
That debt is purchased by private investors, and the money used to buy the debt is made available to a social service provider.
If the social service provider delivers services that meet or exceed the goals, the private investors are repaid with a positive return by the government. The government pays this return when it can save money or increase its tax base due to the intervention.

Here is how it worked for JVS in Massachusetts:

“In the case of JVS’s PFS project, the 40 investors will be repaid based on successful wage gains for participating clients, determined by administrative wage data and independent evaluation. The benefit to the Commonwealth of Massachusetts in the short term is to try a new service delivery model at scale without financial risk, and in the long term reap the benefits of increased tax revenues.”

Social Impact Bonds are very expensive, time-consuming, and complex to set up. They can only work if the intervention can have a meaningful financial impact for the government. You need government buy-in, social service buy-in, investor buy-in, and independent auditor buy-in. But, they can be very effective.

Wealthfront and Betterment are robo-wealth managers that help people easily and passively invest their money. Through my work on Talk Hiring, I know that money is often the greatest constraint in a nonprofit. I wish there was a website where I could passively invest money into social impact bonds, and if they work, I’ll earn a return. I’m sure I’m not the only person who would be interested in something like this! If you find this interesting, feel free to reach out to me.

Reducing police funding

I’ll be supporting the cause of many community-based organizations to push for greater investments in social services and reductions in police funding. I’m not in favor of entirely gutting the police force, but I think that we should think hard about the trade-offs of every incremental dollar that we spend on policing instead of healthcare, education, mental health, transportation, etc. Police are first responders to calls of domestic violence, homelessness, and mental health; social service professionals might be more effective. Higher wages result in less crime. Amidst COVID-19 budget shortfalls, many supportive programs are being cut without any cuts to policing.

In the NYC area, I have been reaching out to my local City Council member, learning about how the budgeting process works, preparing to vote, and supporting local organizations. In all honesty, the process of learning how local budgets are made, actually taking the time to know the name and values of my local City Council member, and educating myself on the upcoming local elections has been eye-opening. If you believe in supporting the cause, check out this Google Sheet: https://docs.google.com/spreadsheets/u/0/d/18pWRSu58DpENABkYUJlZw1ltCPZft7KJc6lFaOZK8-s/htmlview#gid=804939284.

As Bryan Stevenson said in Just Mercy, “the opposite of poverty is not wealth; the opposite of poverty is justice.” I hope you’ll join me in trying to do more.

User testing and user feedback

2020-05-25T00:00:00+00:00

Over the past two years, I’ve tried some relatively unconventional strategies to user test Talk Hiring. I’m proud to have worked on a product that is so easy to use that people who claim to be “bad at computers” can still use it.

First off, a book.

I recommend The Mom Test as a good primer on how to elicit feedback from users without biasing them. Or as the author describes it: “how to talk to customers & learn if your business is a good idea when everyone is lying to you.” At the bottom of the website linked to above, you can enter your email and get some of the book content for free.

Face-to-face guerrilla testing strategies:

Train stations. I’m not talking about Penn Station, Grand Central, or the like. I would go to my local suburban train station that has 2 tracks. Commuters waiting at the train are as captive an audience as they come. Plus, with train schedules, you can figure out how much time they are waiting for. I would head over there when the trains to New York Penn Station and Hoboken would be stopping there within 15 minutes of each other, to maximize foot traffic. It was pretty easy to walk the platform and find people willing to try out a new product, many willing to do it for free. Once I moved to the Bronx, I did the same at the local Fordham train station.

Hiring events. Talk Hiring helps people improve their interviewing skills, so naturally, we found good user testing subjects at hiring events. Specifically, I wanted to target very busy hiring events where job seekers had to wait to talk to employers. Suburban job fairs are relatively empty compared to NYC job fairs, where the lines can easily snake around the block for the opportunity to talk to a handful of employers. I would walk the line and ask bystanders if they wanted to test out whatever it was that I was working on. People waiting on line at a job fair are feeling a combination of anxiety and boredom, and offering them the opportunity to try something out was welcomed by many.

Stopping people on the street. This did not work well at all for me. Some may have more luck at it, but I found that people hate the intrusion because they’re always in the process of doing something or getting somewhere. In my neighborhood of the Bronx (Morris Park), I also found a lot of people didn’t speak English. The Bronx has a hispanic majority, but I do think some people used this as an excuse to avoid me.

Paying people for participating in user tests. Having a bunch of money in my back pocket seemed unsafe, so instead, I had an assortment of Amazon and Uber gift cards with different denominations in my backpack. In NYC, some of the older folks didn’t really use Amazon, but they did use Uber to get around in emergencies.

Online user testing strategies:

Userbob (https://userbob.com/). If you have something online that people can test from their home, Userbob is the most affordable online user testing site I’ve found. They charge $1/minute, and they will recruit the study participants for you. If I wanted some quick feedback on something I was building, I could spend $25 and have 5 people spend 5 minutes testing it out within a day or so. It’s flexible to the minute in terms of time increments. They share screen and audio recordings as people go through the steps in the user test.

Collecting user feedback:

Quick Zoom screen share calls with users as they encounter problems. A few months ago, we integrated live chat into Talk Hiring. When users reach out about a problem (and I have a free moment), I try to hop on to a Zoom call with the user ASAP. This has been incredibly helpful because I can see exactly what is confusing or broken to the user, I can ask follow up questions, and we can strategize which product changes could help future users. I’ve made numerous small changes to our product because of these quick Zoom screen share calls. I can learn a lot more from a screen share than from a live chat conversation.

Weekly calls with pilot users. When Talk Hiring was in the early stages, we ran pilots with job training programs. One of the requirements for running a pilot was that we had to have weekly phone calls with someone at the program to get quick and constant feedback on what was working and what wasn’t. I also used these calls to talk about the new bugs we fixed and features we shipped in the past week. Users love hearing when you build something that was their idea.

User surveys. After every mock interview, users can fill out an optional 3 question survey. We automatically tag the feedback as coming from a specific user and career readiness program. We’re then able to look for trends in our feedback across organizations and within organizations. This is pretty common stuff, so I won’t go into too much detail here.

How much should it cost?

2020-05-20T00:00:00+00:00

Talk Hiring sells its mock interviewing tool primarily to nonprofit job training programs, and more recently, high schools. There are playbooks online on how to price online educational tools for schools, but not as much for nonprofits. I’ve spent many days squirming around, trying to figure out the optimal strategy for pricing the tool, and I thought that walking you through my pricing evolution may be helpful.

June 2019: No Longer Free Forever. We offered custom flat-rate discounted pricing based on the number of potential users. New programs were offered 6-week free trials.

August 2019. No free trials. Instead of free trials, we offered 60-day money back guarantees and continue to price based on the number of potential users. It was the early days of charging for Talk Hiring, and I wanted to work with only serious prospects with budget. For the programs that decided that they didn’t like the product, they could effectively run a free trial with us with our money-back guarantee.

Why we moved away from this pricing:

Although we did bring on 2 paying customers with this pricing, it became obvious pretty quickly that this was not how nonprofits wanted to buy software. Finding money for anything in a nonprofit can be a battle, and asking for money before we had proven our value was a hard sell.
Larger nonprofit job training programs were not willing to agree. Contracts in the $5,000-$15,000 per year range were too significant financially for these programs to be comfortable essentially loaning us a big chunk of change for 60 days while they test out this product. For larger programs, I could have offered a more nominal per month cost when starting out. That may have helped a little, but we still would have been facing the obstacle of bullet #1.
Talk Hiring was (and still is) a new business and many nonprofits are risk-averse, rightfully so. A money-back guarantee is only good as long as Talk Hiring survives, and some potential customers were not 100% convinced that they would be able to get their money back.

Once Talk Hiring is more mature, I would like to revisit this pricing strategy. It works well when potential customers have a high-degree of confidence in your business and the price that they’re paying while testing it out is relatively low.

October 2019. Free Trials + Usage-Based Pricing with an Upper Ceiling.
We had a price calculator on our website where programs could enter the exact size of their enrollment, and get monthly and annual price quotes for their program. We changed our pricing to offer $10 per mock interview, with an upper ceiling of $20 per user per month. The upper ceiling was to give customers the assurance that the cost would never be above (# of users) * $20 in a given month.

Why we moved away from this pricing:

Although this is a common way to price software, asking a job training program to know the exact number of people that would want to use a mock interviewing tool is next to impossible. Plus, many job training programs support low-income populations that are more transient in nature.
For budgeting purposes, nonprofits really want to know how much something is going to cost. Of course, they also want the best price that they can get, but all else being equal, flat rates are preferable. The price could swing by hundreds or thousands of dollars from month-to-month with usage-based pricing.
Usage based pricing works well when the price is very low (think AWS charging cents per hour of server time). We don’t want instructors thinking, “should I do another lesson with Talk Hiring this month or save the money for something else?”

November/December 2019. Pay What You Want Pricing and Tiered Pricing.
Pay What You Want: I read a book by Tom Morkes on Pay What You Want pricing one Saturday morning and was intrigued. Nonprofits (especially religious institutions and museums) use this pricing strategy all the time. Plus, I was getting tired of trying to figure out how to price this tool, and thought, why not dare my customers to tell me how much they would pay for it. Given that my customers know me well, I thought that it would be awkward for them to decide to pay $0 or something only a little bit above $0. I also thought it could help generate good will amongst my customers.

Tiered Pricing: I talked to as many financial professionals at nonprofits as I could (CFO, VP of Finance, etc) to learn how they like to purchase software. The overwhelming majority wanted a predictable, good price (with a nonprofit price discount).

How I tested these two pricing strategies:

A/B tested an email campaign to different prospects. Option A was our flat-rate, tiered pricing and Option B was pay what you want.
I reached out to a few recent prospects that I had talked to, and offered them pay what you want pricing. Pay what you want didn’t motivate any of them to move forward, which really surprised me. Of course, there are many reasons why they may not have moved forward, but given that these were prospects with real interest in the product, I was surprised.
I reached out to Tom Morkes for his advice on using pay what you want in a b2b scenario. He said that pay what you want works best when a single seller is working with a single buyer because it requires the human element to work. Selling to committees or groups removes that human element.

We still run tiered, flat-rate pricing to this day.

April 2019. Simplified, tiered pricing. I came to a few realizations:

We had too many pricing tiers. We had 7 different pricing bands based on the number of participants that a program worked with in a given year. Most of our customers were paying $100-$200 per month or $400-600 per month. We should simplify and charge the midpoints of our most popular pricing bands: $150 or $500 per month.
The number of staff that want access to our staff dashboards scales with the number of users in the program, for the most part. As I mentioned before, estimating enrollment in a job training program is much harder than estimating the number of staff that will be involved in prepping job seekers for interviews.
COVID-19 was directly impacting job training programs (skyrocketing unemployment paired with tenuous funding). By pricing based on staff dashboard access only, we are able to open our doors to as many job seekers as programs need to support during this time of crisis.
My friend Soroush shared this article with me on pricing best practices. I’ve read many articles on the topic, but I found this one especially concise and easy to implement. It talks about price anchoring, the decoy effect, and the paradox of choice, and I used all of these tactics in my new pricing.

As of writing this post, here is a screenshot of the Talk Hiring pricing page.

I’m sure in a few months, I’ll have a new take on our pricing, but for now, I’m happy with it!

Final Thoughts: Flat-rate tiered pricing is the norm for selling SaaS products, but the real challenge is figuring out what the different price points and package features should be. I don’t regret the winding road that took me to this point. By experimenting with so many different pricing strategies, I was able to learn what works and what doesn’t work for Talk Hiring’s customers.

Transitioning from free pilots to a paid product

2020-05-14T00:00:00+00:00

We launched our mock interviewing tool in late January of 2019. This initial product was pretty basic, but we had honest interest from a handful of job training programs. We ended up running about 10 free pilots over the course of 4-5 months. Transitioning from free pilots to a paid product was messy and awkward.

Here’s a photo of me doing a pilot kickoff at Dress for Success Northern NJ:

I got into this business by thinking that voice is a much better interface than text for many job seekers. I still think that. My plan was to give out our automated mock interviewing tool for free, provide value to a lot of job seekers, and then bring on employers to start connecting them via an online recruitment marketplace. But within 4-5 months of running free pilots, I realized that my strategy likely wouldn’t work. Why?

Convincing and then onboarding organizations to a new product is a slow process, and as a self-funded business, I needed to get to breakeven as quickly as possible. At the Next Gen Summit, David Rose, a prominent angel investor, said that the top 5 rules of entrepreneurship are all the same: don’t run out of money. If we wouldn’t be paid until enough job seekers were using Talk Hiring for employers to be interested, we wouldn’t be able to even attempt to generate revenue for an additional 6+ months. By charging for the product now, I would be able to de-risk the financials while still leaving the future goal of working with employers on the table.
We were able to sign on some national programs that work with thousands of job seekers as pilot partners. But, every large program understandably wanted to pilot the tool with only a segment of their organization. Even if we ran successful pilots, our growth within organizations would be slow moving.
It was an awkward process for our pilot partners to run a free pilot for a free tool. We set up the pilot process really as a way to learn about how to make the product better in a controlled setting. But, it’s hard to really take a pilot seriously when, after the pilot, the pricing is $0. By charging something for Talk Hiring, partners take the free trial more seriously because they know that they’re going to have to gather strong evidence to convince the team to eventually buy this tool.

When did we transition from free pilots to a paid product? Once we had testimonials from happy users, a product that a few job training programs seemed to be getting value out of, and a small study to measure the effectiveness of the tool.

Why was it awkward/messy? I thought that the Talk Hiring mock interviewing tool would be free forever, and all of our pilot partners did too. I had to communicate why we were changing our strategy with each pilot program, inform all upcoming fall pilots of this change, and figure out how to price the product for the innovators that took a chance on me and this brand new tool. About half of the programs that had really used the tool during the 3-month pilot program paid a (very discounted) price.

How Talk Hiring got its first users

2020-05-10T00:00:00+00:00

Talk Hiring (talkhiring.com) is an automated mock interviewing tool. To date (May 2020), we’ve conducted over 3,000 automated mock interviews with job training programs, high schools, and colleges. I’m the Founder/CEO and have been working on this business full-time since August 2018.

Talk Hiring started off as an automated phone screening tool for employers. We simply wanted to help employers save time and speed up their hiring process by automating their initial phone screens with job seekers. This business idea didn’t last long (<5 months).

To find businesses that might find our tool useful, I started by pounding the pavement and walking into small businesses in NY/NJ. Although it’s free to do, I wasn’t very successful (however, this did connect me to Talk Hiring’s first employee, Anson Ervin). Some reasons:

I hadn’t thought through the fact that I should have professional business cards. I was ripping off sections of paper from my notebook with my contact info. Those “business cards” had a response rate of 0%.
The manager of a store/restaurant is often not present (or maybe they were lying to me).
Most of these small businesses don’t do lots of hiring, so automating job interviews isn’t that exciting.
The second that a customer walks in, I was less important.
Nobody likes walk-in solicitors. I distinctly remember being yelled at more than once.

A replica of my initial business cards :)

Other strategies that didn’t work:

Applying to jobs online but using a Talk Hiring flyer as my resume. I got no bites and was surprised how many times I got calls from recruiters about how great a fit I was for the position.
General networking groups. I got 2 free pilots out of networking groups, but most members are solopreneurs/extremely small businesses (< 5 people companies). I met some great people that are very helpful to the business to this day, but it just wasn’t a great customer acquisition channel.

On a whim, I decided to check out a job fair in Trenton, NJ. That was the start of our job fair strategy. Some job fairs are ideal places to find initial beta testers of an HR tech product. But I stress some. The best job fairs for getting face time with hiring managers are free ones (run by the government) located in suburban areas. Job fairs in NYC are overcrowded with lines snaking around the block. You want a job fair that isn’t too busy so that hiring managers are bored enough to talk to you 🙂. And, not surprisingly, free job fairs have the most employers. I’m a Jersey kid, so this job fair website was my bible (got to love Mapquest directions for every event). I would walk in with a stack of resumes in a folder as to not look suspicious, and have a stack of Talk Hiring flyers below.

I was able to pilot our first product, an automated phone screening tool, with a custom kitchen range hood company, a lead generation company, a home health aide business, a gym, a chess education company, a car wash, a day care, and a few restaurants.

At these job fairs, I would meet people that worked at job training programs. Job training programs are always looking to enroll unemployed/underemployed individuals in their programming. Plus, I knew a lot about this industry through volunteering with a job training program in Brooklyn for a year.

I heard “can we use this tool for interview practice?” a few too many times at these job fairs. I decided to run an experiment and reconfigure the tool into a mock interviewing tool. We’ve been focusing on our mock interviewing tool ever since.

So you want to run a study?

2019-07-01T00:00:00+00:00

We ran a study to measure if using the Talk Hiring mock interviewing tool improves interviewing skills, and we have proven that it does! Here’s a link to our study setup and results. Running this study was much harder than I thought it would be. Here are my learnings for others who might want to run their own study!
Why did we run a study?
Our product helps job seekers become better at interviewing. To make a claim like that, we wanted to back it up with data. First we thought, why not measure hiring outcomes? There are too many confounding variables to measure hiring outcomes from people who use or do not use our tool; plus, it would take much longer to prove or disprove. Then we thought, can we use our existing interview data to measure interviewing improvement? Users practice with over 100 different interview questions across different industries. It’s hard to compare interview quality across different questions, let alone across different industries. We decided to run a study so that we could control variables and prove the worth of our mock interviewing product. Recruiting Participants:
As the operator of a self-funded startup, I initially tried to run the study without paying participants. Talk Hiring works with workforce development organizations mostly in NYC, and I tried to recruit participants through those organizations. We were able to recruit a few people through those channels, but the study would have dragged on for months if we had continued down that path. I tried running free mock interviews with people, and using those interviews as data points in the study, but that was taking up so much of my time with lots of people canceling/rescheduling. I finally decided to recruit for the study via a $10 Craigslist posting and through my existing Talk Hiring user base.

A Note about Craigslist:
If your participants can complete the task remotely, always set the location of your Craigslist task to a major metropolitan location. We had people from all over America completing our study because they were searching for paid studies in Manhattan.
Money: I paid every study participant $5 via PayPal. Even for that relatively small sum of money, people really needed the money. I tried to pay everyone at the end of each day, but I got pushback. A few people told me that they were planning on using my $5 for lunch that day. Others would email me in all caps right after they completed the study, asking when they would get paid. Some told me that they never received the money, and I had to screenshot my PayPal receipt to show them that they had been paid. Interacting with so many people who needed my meager $5 was disheartening. Choosing the Right Significance Test: There are lots of statistical tests of significance, and it has been 10 years since my high school stats class. I did more than my fair share of Googling to figure out how to structure our study and which significance test to use. Initially, my plan was to use a one-sided z-test to measure if a participant’s interviewing scores improved by using the tool. Once the study was underway, I realized that it was not possible to accurately score interviews on a 1-5 scale. Instead, I decided to measure simply if a participant’s 3rd try at an interview question was an improvement over their 1st try, defaulting to “no improvement” if it wasn’t clear. I ended up using a single proportion t-test to measure statistical significance. Unexpected benefits: 24 people participated in our study. Out of the 24 participants, 2 people sent me their resumes for employment opportunities, and 4 people told me that they would spread the word about our mock interviewing tool!

A special shoutout to Joyce Cahoon and Sam Waters for sanity checking all of my statistics in this study!

Foundations of Deep Learning

2017-03-11T00:00:00+00:00

First blog post on the x.ai blog about Activation Functions in Neural Networks. You can read it here on Medium or here on the x.ai blog.

Ruthless Prioritization

2017-02-19T00:00:00+00:00

Prioritization is one of the most complicated, contentious, and crucial aspects of building a startup. There’s always an order of magnitude more tasks that your team can take on. Making sure that your team is working on the highest value work items is vital. HBR (Harvard Business Review) just released a phenomenal article on prioritization. I’ll highlight the key points here and analyze them by pulling from my startup experience.

The 3 Interdependent Variables in Prioritization “There are three interdependent variables that are essential for executing any initiative — objectives, resources, and timing. You can’t produce the desired effect of a project without precise objectives, ample resources, and a reasonable time frame. If you push or pull on one leg of this triangle, you must adjust the others. All three variables are important, but resources reign supreme” This makes sense of course, but is very easy to forget. By speeding up a project, you’ll pull on more resources and clearly alter the importance of certain objectives. If you shift objectives, you’ll shorten the timing for some initiatives, but increase the timing for others (while adjusting your allocation of resources). If you hire or fire employees, the timing of your projects and the importance of objectives will necessarily change. To engineers, this will likely remind you of the CAP Theorem. The reason why resources “reign supreme” is because they are the leg of the three legged stool that actually gets things done. Without having dedicated resources, objectives and timing just don’t matter.
This is especially true in startups because the team is tiny. A few crucial team members leave the company, and management has to quickly shift priorities and timing of projects. Similarly, making the hiring plan more aggressive causes very obvious and immediate changes to what the team can get done in a quarter.

The 3 Levels of Priority: Critical, Important, and Desirable “A critical priority is an objective that must be successfully accomplished within a specified amount of time, no matter what.” The “no matter what” bit is the key part. Critical projects should be given unlimited resources towards achieving their goal. The manager running the project is allowed to pull resources (people and money) at will to get the project done because the timing is fixed. These are the classic deadline heavy projects. “An important priority, on the other hand, is an effort that can have a significant positive impact on performance. For these initiatives, resources are fixed and the variable is either time or the objective” Important objectives are still objectives that have to get done, but the crucial part is that there is no strict time or objective pressure. “A desirable priority is an effort in which resources and time are both variables. The organization desires an outcome but cannot absolutely commit specific resources over any specifiable time period.” Desirable objectives would go under the “nice to have” category. These could be optimizations to the current product, infrastructure improvements, etc.

How Resources are Affected It becomes crystal clear that in this 3 tiered priority system, managers must be very careful about assigning tasks as critical. Critical projects can take on unbounded resources. They can’t pull resources from important projects; those still need to get done. The only option is for critical projects to take on resources from desirable projects.
At a startup, it’s very easy to throw around the word “critical” when we actually mean “important”. The word critical is very clear to employees, and it can start to become a catch-all for anything that is necessary to get done. Employees feel urgency to get things done when they’re working on critical projects. But, having too many critical projects causes confusion. Certain projects that are actually important will try to poach extra resources, limiting the pool for truly critical initiatives.
Additionally, the organization can start to feel what I call “critical burnout”. If everything is critical, then nothing is. We become desensitized to what working on a critical task really means. This is a disaster. There will actually be critical things to get done, and it’ll be harder to get your team to mobilize if it feels like just another work item.

Thanks for reading! I consider myself a prioritization junkie, and I’d love to hear about your experiences with this subject and how it relates to your organization.

Link to Full Article: https://hbr.org/2017/02/a-better-way-to-set-strategic-priorities.

Mailbadger: A Postmortem

2016-04-19T00:00:00+00:00

At the start of junior year at Duke, I was working on a startup called MailBadger with my friend, Kirill Klimuk. Essentially, MailBadger was a two-fold enterprise software solution. It helped companies automatically follow-up with their clients, and it displayed these results in a simple, user-friendly interface organized by client or by reminder. MailBadger was going to solve the problem of requiring employees to remind themselves to follow-up with their clients. Instead of reminding themselves to remind others, employees would simply set a reminder with MailBadger for their clients, and MailBadger would send an email reminder at the specified date and time.

We were interested in helping accountants that handle taxes. Many accountants that we talked to echoed the organizational challenges of being an accountant. During tax season, it is very time-intensive to make sure that all of an accountant’s clients have sent the accountant all of the legal forms that the accountant needs to do the taxes. What often happens is that a secretary will keep a huge Excel document with client names on one axis and documents required on the other axis. Then, the secretary will mark off which clients have submitted which documents. When clients do not submit documents in a timely manner, the secretary has to constantly follow up until the client has sent all of the required documents.

Kirill and I saw accounting as a huge opportunity to digitize the receipt of documents and automate the following up with clients to send required documents. However, we realized very quickly that CPAs are slow to change, are not particularly skilled with technology, and value the personal connections that they have with their clients. Many accountants are Excel power users, and do not like using products that are not tied into Excel. Additionally, because many accounting firms are run by multiple partners, it can be difficult to get all of the partners to agree on a new technology product to implement throughout the company. Finally, many clients stay with accountants because of the personal connection that the clients have with their accountants. Some accountants saw MailBadger as a product that would make communication less personal, and could translate into lost clients at an accounting firm.

word2vec in JavaScript

2016-04-14T00:00:00+00:00

One of my personal goals is to better understand deep learning. Word2vec, an algorithm developed by Google, is one of the simplest (but still not simple!) forms of deep learning. Deep Learning is a type of machine learning when there is at least one hidden layer. Word2vec is an algorithm that develops vectors for words based purely on the assumption that similar words are often used in similar ways. By running words through this algorithm, you can find a vector for a word, and then take the cosine between 2 vectors to find how similar the words are…so cool! There are 2 ways that word2vec can be implemented.

CBOW (Continuous Bag of Words) - The algorithm is given a context (n words before and after the word), and optimizes for the middle word. This is the strategy that I used and implemented.
Skip Gram - The algorithm is given the middle word and optimizes for the context

The gist of the CBOW word2vec algorithm is the following:

Start with an enormous corpus. An often used corpus is called text8 (http://mattmahoney.net/dc/textdata and search for text8.zip). I copied it into corpus.txt by using head -1 text8 > corpus.txt. This command puts everything on the first line of the text8 file into corpus.txt. text8 is a huge 1 line file, so it works out.
Assign random values between [-1, 1] for all of the entries in W, W’, and h.
Generate a map D and DPrime. D is a map with every word in the vocabulary as the key and with a list of each context that it appears in as the value. DPrime is a map where each key is a word and its value is a list of some contexts that DO NOT appear in the text for that word. When Google was developing word2vec, they found that it was important to both optimize words and contexts that SHOULD go together, as well as “teach” the network about words and contexts that should NEVER go together.
Run each context in D and DPrime through the network below TRAINING_ITERATIONS times.

The architecture of the one hidden layer network: * There are V unique words in the vocabulary and the hidden layer (single dimension vector h) has length N
* With CBOW, vector x becomes a vector with a 1 in all places where the context word exists and a 0 everywhere else.
* In my script, I arbitrarily used an N of size 100. All other constants that I chose are at the top of my script.js file as well.

In each iteration of the learning loop:
1. Create intermediate vector Vc:
  1. The x vector is 1 at the indices of the context and 0 everywhere else. Therefore, when you multiply W transpose * X, the output will be the columns of W transpose that have the indices of the context. This is equivalent to the rows of W that have the indices of the context words. Vc is a vector with N rows and 1 column because that is the output of doing W transpose * X
2. Create intermediate vector Vw:
  1. Vw is the column of the W’ matrix (or the row of the W’ transpose matrix) that corresponds to the middle word
3. Find the intermediate output. This is the output that you are optimizing in each iteration of the loop. The following equations come from taking the derivative of the Maximum Likelihood Estimation.
  1. If we are looking at a middle word and context that is not in the corpus:
    1. Log(1 / (1 + e^(Vw x Vc)))
  2. If we are looking at a middle word and context that is in the corpus:
    1. Log(1 / (1 + e^(Vw x Vc x (-1))))
4. Use gradient ascent to update each element in the W and W’ matrix that was affected by this iteration. The rows of W that are affected are the rows in the context, and the column of W’ that is affected is the column corresponding to the middle word. Gradient ascent is used to maximize a function, and in this case we are trying to maximize the likelihood that the middle word is the word in between the context. Gradient descent is used when trying to minimize a function.

In every epoch of the loop, I update all of the word vectors by writing to a csv file, as well as writing a hello.txt that finds the 10 words that are most similar to the word hello. We find word similarity by finding the 10 words with the largest cosine similarity.

I hope my JavaScript is simple and easy to understand 😃 https://github.com/harrisosserman/word2vec

The best materials that I found on the topic are these 2 YouTube videos from University of Waterloo Professor Ali Ghodsi. Thanks so much for your lectures! https://www.youtube.com/watch?v=TsEGsdVJjuA https://www.youtube.com/watch?v=nuirUEmbaJU

After writing the algorithm, I enjoyed reading this paper to tune and optimize my word vector output: https://levyomer.files.wordpress.com/2015/03/improving-distributional-similarity-tacl-2015.pdf

Alarm Clock iOS App

2015-07-07T00:00:00+00:00

I share my bathroom with a roommate. Nobody likes to wake up and find your roommate in the shower. To solve this problem, I built an Alarm Clock app that shares my wake up time with my roommate. That way, I can know when my roommate is planning on waking up, and I can set my wake up time accordingly.

I use UILocalNotifications to do the wake up. These notifications work great because an internet connection is not required for the notification to fire and I don’t need to manage running a background job. To do authentication, I used Twitter’s Digits SDK, which works seamlessly for doing phone number based authentication. Having an email address and password are not required to create an account on my app. For all my database storage, I used Parse. I did not want to manage a server for this small application.

All of the source code is here: https://github.com/harrisosserman/AlarmClock.

NSNotification Debt

2015-05-19T00:00:00+00:00

Disclaimer: NSNotifications are not iOS push notifications

NSNotifications are signals that can be passed globally between classes in an iOS project. Classes can send (or post) notifications to classes that listen for the notification. For more information, check out the documentation here

Extensive use of notifications are bad for a number of reasons:

Notifications lead to poor software architecture When building an iOS app, it’s very tempting to use NSNotifications to pass around signals. Just because it is easy doesn’t mean that you should use them often. Notifications are easy to use because the sender does not need to hold a reference to the receiver. Therefore, your app could be poorly designed (from a software architecture point of view), but held together by crisscrossing notifications.

Each notification can become gradually larger scoped than was initially desired It’s critical that each notification’s usage is defined. Otherwise, it is very easy for a notification to become used for more than it was designed for. For example, let’s say there is a ‘userChanged’ notification that would get fired every time that the user’s information was changed. A name like this is often too vague because a lot of different things can change for a user. Therefore, you will probably end up reloading/recalculating more parts of your app than are necessary, hurting the app’s performance. A better name would be something like ‘userChangedContactInfo.’

It’s hard to debug an app with lots of notifications When a notification is fired, more than 1 method can (and often is) executed. Multiple code paths need to be investigated, which is annoying and time-consuming.

My First Business Ever: KoalaCab

2015-01-16T00:00:00+00:00

During the summer after my freshman year at Duke, my friend and I started our first business. I built my first website. It was a taxi sharing and taxi booking website called KoalaCab (I know Uber now does this, but it was pre-Uber). We ended up booking 450 cabs during our first year of operation.

Colleges are great places for taxi sharing because:

Students are price sensitive and want to spend as little as possible
Students are generally coming from/going to the same places (especially the airport before and after school breaks)
Students want to meet new people

This was our slogan and may have been the best part of the business:

Building the Website At the end of freshman year, I had written very little code before, and had no idea how websites worked. I saw a tutorial on Joomla called Website Essential Training, and it taught me the basics of how to build a website with PHP and MySQL. Here is version 1 of the website: I ended up using CodeIgniter, a PHP MVC framework, which helped with security (specifically SQL attacks) and code cleanliness. I used JQuery Mobile for the KoalaCab mobile website. In hindsight, building an app instead of a mobile website would have been the right decision, but I didn’t have the time to learn iOS/Android development.

Final version before I killed off KoalaCab:

Building the Business For KoalaCab to work, we would need to partner with a taxi company in the Duke area. Riders would book rides through our system, the taxi company would drive the passenger, and we would take a cut. We met with all the taxi companies in the Duke area to see what they would offer us. All the taxi companies were willing to partner, some on better terms than others. One company was willing to give my friend and me free rides for life (in hindsight, maybe we should have done that deal). We vetted taxi companies on their tech capabilities, their management style, promptness, and reputation among Duke students. We ultimately chose Charlene’s Safe Ride because they allowed students to pay with their FLEX school dollars.

Learnings College students are very last minute, which is terrible for a web-based taxi sharing service. It’s very hard to convince students to book a cab in advance. The later the cab is requested, the harder it is for people to find out about the ride and join in. When users came to our site, they were forced to first search for a cab with a similar itinerary before booking a cab. If a cab was already doing a similar route, the user could join in. If not, the user could create a new ride. If everyone is searching/booking last minute, it is harder for people to truly cab share, and KoalaCab becomes more of a cab booking website for solo rides.

And please don’t be in “stealth mode” while building your company. Nobody wants to steal your idea. Tell people about it, and have potential customers test it constantly.

Scaling Property of Randomness

2015-01-15T00:00:00+00:00

I really enjoyed the book Fooled by Randomness by Nassim Taleb, and I came upon many interesting statements about randomness in markets, the most interesting one being the scaling property of randomness.

In the stock market, the general direction that stocks move in are for a reason, but the second or millisecond deviations that stocks have can be mostly attributed to noise or randomness. As Taleb did in his book, let’s take a stock that has a 15% return with 10% volatility per year. This translates into a 93% chance that the stock will go up in that year, assuming that the noise can be approximated by a normal distribution.

At a second time interval, there is only a 50.02% chance that the stock will go up. At a day granularity, there is a 54% chance that the stock will go up. At a month granularity, there is a 67% chance that the stock will go up, and at a year granularity, there is a 93% chance that the stock will go up. When I first read this, I was shocked. Randomness does not scale linearly, and therefore when analyzing the markets at a small time scale, stock price variations are often meaningless. This means that at the smallest of time scales, markets are inefficient, a very interesting conclusion.

Path

2015-01-15T00:00:00+00:00

Path is a mobile-only social network for your closest friends. It provides the basics of a social network (messaging, news feed, picture/video posting, location check-ins, status updates, and “friending” of people). But, because it is more intimate than a larger social network (ex. Facebook), it has become a much richer and detailed way to document my life.

Things I Love Path is known for its design. Path’s best design elements are the clock, which changes as the user scrolls, and the “+” button that allows the user to add a new moment. Moving clock hands as the user scrolls is a truly innovative UI element, and is something that I have not seen on any application. Pressing the “+” button beautifully releases five different buttons that the user can select.

My favorite features in Path are emoticons, and book/movie/music/tv show sharing. Path allows emoticon posting on moments. Unlike Facebook where users are only able to ‘like’ a post, posts (or moments) on Path can be smiled at, winked at, frowned at, surprised at, or loved. This allows for rich and quick responses to a friend’s moment, while adding a unique feature to social networking.

I really enjoy book/movie/music/tv show sharing. My friends have shared lots of great media through Path that I did not know about, and I have discovered new media this way. Path makes it easy to share what music is currently playing on a user’s smartphone, and has autocompletion for most music, movies, books, and tv shows. When friends share songs they are listening to, other friends are able to listen to a preview of the song in the app, and when friends share movies or books, Path displays a short description of the movie or book in the app. The intimacy of Path promotes more sharing than on Facebook, and has allowed me to better stay in touch with my closest friends.

Things I Don’t One feature I never use is the ability to share sleep information. I have shared sleep information with friends, and every time friends either comment on me sleeping too much or me sleeping too little. The predictability of the responses to my hours of sleep gets old, and I no longer record my sleep data. I have approximately 30 friends on Path, and none of them share sleep data either. Sharing how many hours you sleep every day seems to be oversharing, and is an unnecessary feature of the application, in my opinion.

A Way to Monetize Path Path is currently trying to monetize by selling virtual goods, either as a monthly/yearly subscription or on a per item basis. This is a great start, but I am skeptical that this tactic will make Path enough money. I believe that sponsored stickers would improve the user experience while making Path and advertisers happy.

There are limitless options for sticker packs, and these would make the app much more fun while bringing in more money for Path. Sponsored sticker packs would be free for users, and would add more sticker variety for users to post. I know that Path is trying to build a social network without ads, but I think that these ads would not be intrusive at all.

My Favorite Books of 2014

2015-01-15T00:00:00+00:00

I made a lot of time to read in 2014. Here were my favorite books:

Hatching Twitter — Incredible read about the dysfunctional founding team of Twitter. I couldn’t put it down…definitely a page turner
The Rosie Project — Fictional book about a nerdy guy who tries to find love by giving out surveys to bachelorettes. He tries to apply the scientific method to love and fails, but in the process, finds love unconventionally.
Ending Aging: Breakthroughs That Could Reverse Human Aging In Our Lifetime — Aubrey de Grey argues that our view towards aging is wrong. We strive for cures to cancer, diabetes, stroke, etc, but are fine watching our bodies age. Aging is just an accumulation of damage in the body. Aubrey gives lots of examples of how we could stop aging, as well as the status of those medical breakthroughs.
10% Happier — Dan Harris, a TV anchor on Good Morning America, had a panic attack on air. He decided to get his anxiety under control with meditation, and claims that meditating every day makes you 10% happier…I agree
The Goldfinch — An orphaned child steals an artistic masterpiece. He travels around America, from NYC to Las Vegas, having life-defining experiences while always holding on to his beloved Goldfinch painting.
Throwback — Jason Kendall, an MLB catcher, explains baseball from an insider’s angle. I enjoyed reading about what the catcher/pitcher signs and signals actually mean.
Waste: Uncovering the Global Food Scandal — Everyone knows that there is food waste, but I never understood just how much food is wasted in all steps of the food production process. Farmers waste crops when they don’t look the way consumers expect them to. Food production plants throw out heaps of excess production when supermarkets won’t take their supply. Supermarkets throw out food past the sell by date, even though the food is safe and edible.
Schulz and Peanuts — Howard Schulz, the inventor and creator of Charlie Brown, struggled mightily in his early life. After each failure, he kept drawing, eventually resulting in the most successful cartoon series ever.