I’ve been working on a startup project on the side for almost a year now – focusing on pattern recognition, natural language processing stuff, and predictive statistical modeling… it’s been fun. At the core, we’ve put together a language analysis engine which looks at a chunk of text and figures out if it’s positive or negative. In researching this as a problem, we’ve determined that if you take three individuals, and then have them categorize the same random text (blog, article, website, tweet, etc) they will agree 63% of the time. There’s a little bit of variance depending on what’s shown, but plus or minus a couple percentage points, is about how accurate a human is. We’ve gone through several different models in doing the predictions, and tweaked the algorithm quite a bit over multiple different versions, but we recently hit a pretty major milestone – we’re now rating articles, or our engine is, with a 70+% accuracy rate. In other words, if we rate something as positive (meaning the author felt positive about whatever they were writing) 70% of the time, the human will agree with how we rated it. ^_^

We’re better at determining human opinion than the average human is.

We’re going to be going into beta soon, on a service that will allow you to track how positive or negative your brand is, by tracking the mentions on the internet – effectively doing sentiment analysis and tracking; if you’re interested, you can sign up here. You can read more about the project in general at www.entrenza.com.

Thanks to Steve & Jesse & Ben, my co-collaborators on the project for making this happen!

For the last year or so I’ve been working on a tech startup with some friends. In doing so, I’ve gotten to work with some pretty cool stuff, and I thought I’d make a list of some of them. Basically, I wanted to extole the virtues of working on a startup as a great way to get real-life practice projects to work on – I have every expectation that we will have at least some success, but even if it ends up being a failure, here are some of the projects that I’ve gotten to work on:

• wrote an spidering application
• setup a mysql cluster
• setup zimbra
• researched several virtualization options
• setup apt-proxies
• wrote a custom smtp daemon / parser
• learned a boatload about bayesian analysis and other pattern recognition and predictive tools
• setup joomla
• setup drupal
• setup linux natting/routing firewall
• wrote project plans
• managed & motivated
• learned how to incorporate
• setup zimbra
• setup dnsmask for dhcp/dns masqerading
• setup bind dns & replication
• learned how to motivate people and lead weekly conference calls
• worked on project management
• marketing
• sales
• setup ldap athentication
• setup openNAS

Now – I’ve done a bunch of these things before in previous jobs, but its still good practice and there were several I hadn’t played with before. It’s a great opportunity to learn, and even if it doesn’t end up succeeding, the time I’ve put in will not have been wasted. I’ve become a better programmer, a better leader, and I’ve had a lot of fun doing it.

If you are interested in keeping track of the morale at your company, project, or keeping track of how positive people are about your brand, or a search term, we’re looking for beta customers. Feel free to ping me @nickbernstein on twitter if you think you might be interested.