It’s been a pretty successful system. I don’t worry about huge strides, I just worry about accomplishing, one thing towards each goal each day. So far, ‘ve registered a LLC and and have regularly worked part time on a startup for over a year. We’re right about to go into beta. As for getting in shape, when I left seattle two years ago and got my California license, it listed my weight at 205lbs. I renewed last week and had 185lbs on it. Prior to leaving Seattle, you could add another 20 lbs to that. I no longer smoke.

I don’t know if this will be useful to anyone else, but I’ve had some success with it, and who knows, maybe it will work for you. If you try it out, I’d love to hear from you and see how it goes.

We’re better at determining human opinion than the average human is.

We’re going to be going into beta soon, on a service that will allow you to track how positive or negative your brand is, by tracking the mentions on the internet – effectively doing sentiment analysis and tracking; if you’re interested, you can sign up here. You can read more about the project in general at www.entrenza.com.

Thanks to Steve & Jesse & Ben, my co-collaborators on the project for making this happen!

Here’s the gist: you would log-in to a website, and be presented w/ an “article” – this would be a blog, website, etc. you would then rate it as positive, negative, & so forth. Anyone who rated 1000 articles in a month (each takes about 1-2 seconds) would be eligible to win a prize, which would either be an xbox 360, or a playstation 3.

So: Three questions:

Tomorrow, after going out to brunch with some friends, and then cleaning the apartment, I’m going to come back and start doing some code profiling to look for memory leaks. I’ve done standard debugging stuff before, and learned the basics of code optimization and such in school back in the day, but the fact of the matter is, until I started working on the startup project, I mainly wrote smaller programs – utility scripts, small apps, stuff like that. I didn’t really ever get into situations where I had to think about memory leaks, or applications which would basically run constantly. It’s other stuff too – if you write a script to automate the creation of this or that, or a .net app to generate foundry server iron configs, you don’t need to put it in an architectural perspective. Now I have to think about that kind of stuff all the time. “what happens when this breaks?” – “How can I write this so I can add another server and scale out horizontally?” – I actually think about this kind of stuff when I’m coding now. I’ll scratch whole bits of things that worked because they’ll cause grief down the line.

I think it’s also made me more disciplined. There’s a big difference between doing your job because you know, eventually, if you slack, you’re going to get grief about it, and if you do it well you’ll get rewarded for it – all by someone else, and setting goals and following through on your own. My home page, over the years has been slashdot, popurls, rootprompt.org and a miriad of other websites… this would be on my work browser. Now, it’s (thank you firefox and chrome for having multiple tabs) a google doc spreadsheet of my goals and columns representing dates and actions. Each day I list what I did to achieve those goals. Another tab contains our ticketing system. Another tab contains our intranet site, in which there are a bunch of daily actions that I try to go through. I would never have approached a job like this if I were being paid by someone else. It took doing this on myself to realize the type of mindset and tools I would have to give myself to accomplish these things.

Another thing that I think has made me a better person is our twice weekly conference call – we have a very loose structure for the company – there’s no office, and we use email, ticketing, IM to communicate and conf. calls to go over progress and complete goals. We basically cover what we’ve done, and what’s next. It also is a good opportunity for me to talk on the phone with friends, who all, at this point, live in different parts of the world. Yes, it’s about a shared goal, and a project, but it’s also about keeping in touch with friends, and doing so regularly. I have traditionally been terrible at keeping in touch with people, and I think that this help me in that regard.

Tomorrow is going to be frustrating as hell. Don’t get me wrong, I’ll have a nice lunch, and put the work out of my mind during that part, and I’ll enjoy the call – tomorrow’s sunday, one of the days we do it – but when I start getting into phase II of the code profiling stuff, and looking for circular references and objects that aren’t being collected, I’m going to get seriously frustrated and stressed out. I’m going to hate it. But I’ll make strides towards getting it fixed. And the idea that taking on a project of this scope, and how hard it is, is making me a better coder will give me some solace. The idea that I’ve had to change my thinking in regards to where it fits in the architecture, I think has made me a better technologist, and the discipline and keeping in touch with friends has made me a better person, I hope. Yes, I’ll definitely be incredibly frustrated when half the stuff I’m trying to do ends up breaking things temporarily, but I think, in the end, it’ll be worth it.

• wrote an spidering application
• setup a mysql cluster
• setup zimbra
• researched several virtualization options
• setup apt-proxies
• wrote a custom smtp daemon / parser
• learned a boatload about bayesian analysis and other pattern recognition and predictive tools
• setup joomla
• setup drupal
• setup linux natting/routing firewall
• wrote project plans
• managed & motivated
• learned how to incorporate
• setup zimbra
• setup dnsmask for dhcp/dns masqerading
• setup bind dns & replication
• learned how to motivate people and lead weekly conference calls
• worked on project management
• marketing
• sales
• setup ldap athentication
• setup openNAS

Now – I’ve done a bunch of these things before in previous jobs, but its still good practice and there were several I hadn’t played with before. It’s a great opportunity to learn, and even if it doesn’t end up succeeding, the time I’ve put in will not have been wasted. I’ve become a better programmer, a better leader, and I’ve had a lot of fun doing it.

If you are interested in keeping track of the morale at your company, project, or keeping track of how positive people are about your brand, or a search term, we’re looking for beta customers. Feel free to ping me @nickbernstein on twitter if you think you might be interested.


I am writing you today in regards to the recent information about the torture which has taken place in the name of the American people. The atrocities, a word I do not use lightly, which have taken place *must* be investigated, and this needs to be done in a manner that is as fast and impartial as possible. I believe that there needs to be an investigation done not by Americans but by a international third party such as the UN. This cannot be perceived as a partisan attack against republicans, it is too important, but an investigation and prosecution of this torture must be undertaken. Our country is nothing without the ideals and principals under which it was formed, and this is must be dealt with.

Thank you for your time,
Nicholas Bernstein


If you are not familiar, here is some background information:

• http://www.dailykos.com/storyonly/2009/5/14/731112/-Seymour-Hersh:-Children-raped-on-camera-in-front-of-women-at-Abu-Ghraib.-How-bad-are-these-photos
• http://shatteredparadigm.blogspot.com/2009/02/undescribable-torture-guantanamo.html
• http://www.huffingtonpost.com/2009/04/21/daily-show-takes-on-tortu_n_189356.html

to balance it out, here is a video of baby pigs being cute: http://www.youtube.com/watch?v=FIWf_hc1_TM

Initially I went to carmax and they offered me $2000. Yeeps! I was shocked. Could it really be worth that little? Kelly Blue book said it was at least worth $4k. So I thought I would test the open market and write a quick perl script to give me the average price for an item on craigslist. Here’s how it works and the code is below. It should be really easy to modify for anyone who could use something like this:


./cl_get_prices.pl 2001+jetta

lowest:         1200
highest:        9999
Average:        6134.48214285714


another example where I search for porsche boxster:


nick:~$ ./cl_get_prices.pl porsche+boxster

lowest:         7600
highest:        33100
Average:        16865.6341463415


Anyway the code is below, and I’ll put a link to the actual perl script:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

#!/usr/bin/perl
$wget="http://losangeles.craigslist.org/search/cta?query=";
$wget .= $ARGV[0];
print $wget . "\n";
$html = `wget -q -O - $wget`;
        @words = split(' ', $html);
        foreach $word (@words)
                {
                 if ( $word =~ m/^\$/)
                        {
                                $word =~ s/(\$|,)//g ;
                                if ( $word =~ m/^\d+$/ )
                                {
                                        if ( $lowest eq '') {
                                                $lowest = $word ;
                                        } elsif ( $word < $lowest ) {
                                                $lowest = $word ;
                                        }
                                        if ( $highest eq '') {
                                                $highest = $word ;
                                        } elsif ( $word > $highest ) {
                                                $highest = $word ;
                                        }
                                        $amt += $word ;
                                        $count++;
                                }
                        }
                }
$average = ( $amt / $count ) ;
print "lowest:\t\t$lowest\nhighest:\t$highest\n";
print "Average:\t" . $average ."\n";

Here’s a link to the actual script you can download;. If you find it useful, let me know.

<pre>print “Sport’s ranking: \t ${$result}{sports} \n” ; </pre>

Which, lets face it, is kinda ugly, but it’s really the only good way to do it, really. It should really be better documented, though. Aside from that, as long as you get the back-end math, you’re pretty OK. Just because you’re doing AI stuff doesn’t mean that you’re automatically familiar with how perl handles references to hashes, though. One of my to-do items is to go back and update the perldoc on it. Anyway, with that in effect I’ve gone and updated the database and I’m now able to get positivity over time. This means I’m actually getting closer to building an internet happiness index, and prediciting how “happy” the internet is as a whole. The next steps are:

• incorporate new bayes functions into existing codebase
• add more sources to the rss feedlist scripts
• optimize the blogparser
• put a nice (fusioncharts?) front end together
• add more hardware for doing the catagorization
• get more people to do more training data
• ???
• profit!

Actually, the “???” is pretty well defined, but honestly, this project will have been fun even it it doesn’t make a dime. Anyway, one step closer.

I was fine with this, I was planning on installing ubuntu on it anyway. The problem was that w/in a few days the “p” and “o” keys had gone dead. No problem, I think, I’ll just take it back. I had left the box at my GF’s house, so I asked her to ship it to my apartment. Once I got home to california, and gotten the box, I took it back to my closest best buy in el segundo, ca. and they refused to do an exchange. An exchange. I don’t want my money back, or anything fancy – they keys don’t work, I just wanted to swap it out, no data transfer even, I wiped it for them. I cajoled, I bargained, I tried to explain what an example of bad customer service this was, but to no avail. The manager wouldn’t even come out to see me. ( !! )

In the last year I have purchased (off the top of my head):

• a laptop (alum. macbook)
• several mice
• multiple laptop cases
• two network routers
• several video adaptors
• a printer
• two 160g usb hd drives
• 1 terrabyte usb hard drive
• several video games
• ethernet cables out the wazoo
• a gigabit switch
• a non-gigabit switch
• extended power strips

I’m not going to enumerate everything I’ve purchased, but I buy a *lot* of computer equiptment. I can’t imagine why bestbuy would have a policy in place that would be so strict that it would willingly loose a customer over an *exchange* – I mean, this is exactly the type of service that drove compusa out of business. I spent a bunch of money, value me as a customer. If anyone from bestbuy reads this, this is the resolution I would like: contact the bestbuy in el segundo, and ask them to exchange my netbook. That’s it. Like for like. Otherwise, there’s a fry’s electronics a block away, and I’m sure they’d be happy to take my money.

-Nick Bernstein.

Figure 1

Netapp’s snap restore product is a fantastic tool in a storage admin’s arsenal. It works well. It’s fast, and it doesn’t need to “restore” data, it just makes a previous snapshot the active file system. That said, I keep seeing the same scenario put forth by netapp and various other folks as a use case for snaprestore, and it’s one where snaprestore would be my second choice. The scenario is this: We’re taking hourly snapshots over the course of a day. A virus breaks out. The files on our cifs shares have been compromised. We’re sure that our current data is infected, and we know that at least some of the data was infected an hour ago. Two hours ago is uncertain, but we are sure that the virus hadn’t broken out three hours ago, so we know the file system at this point is clean. The recommendation you always hear is to use snap restore to roll back the file system to three hours ago, and you’re now clean and not infected. Unfortunately, as a side effect, we’ve lost all the data after that point – which, I should mention, is the intended result: if we hadn’t our files would still be infected, right? My solution is to instead clone the volume using a really neat tool called flexclone.

some snapshot basics

I’d like to suggest an alternate method, but before I get into it, it probably makes sense to talk about how snapshots work. The idea is very simple. Each file on a file system has an inode. An inode contains information about a file – the owner, when it was created, etc. If you’re on a windows box, and you view the “properties” of a file, you’re seeing the information contained in an inode. A hard drive is made up of blocks of data, and another thing that an inode does is point to the blocks that make up the data for a given file. If a file is big, the inode won’t actually point to the data blocks themselves, it will point to indirect blocks which then point to data blocks ( inode -> indirect level 1 -> data ). If a file is really big we can have multiple levels of indirection, like in figure 1. A snapshot is, for all intents and purposes, a copy of just that top level inode block. This block then contains pointers to the indirect blocks, and those indirect blocks to the data blocks. ( Is this fun yet? ) Once we’ve made a copy of the inode (parent) the indirect and data blocks (children) are effectively frozen, since a block cannot be changed unless it’s parent, or parents (original inode and now the snapshot) agree. The way we handle modifications to files in the netapp world is to write those changes to a special reserved section of the volume called the “snapshot reserve” (apt) and to update the original inode to point to those blocks. It’s pretty slick, really.

flexclone

In order to understand how this alternative solition works, we need to understand how flexclone works. Flexclone – to over simplify things – is a writeable snapshot. Actually, it’s really not that much of an over simplification, really. If we go back to our snapshot basics where we talked about how each file has an inode which points to data blocks, I should mention that under the hood, everything on a filesystem is actually just, well, a file… even directories. A directory is just a file that contains a list of file names and where the inodes for those files are, that’s the data in that “file’s” data blocks. The top of any volume (think of this as a drive) is a directory, and that directory in turn, has an inode. This one we call the “root inode”. When we take a snapshot, of volume, this is the inode we’re basing our snapshot of. So, what we end up with is: [ root inode ] [ snapshot copy of root inode ]. What we can do is add one more copy to make a “clone” of the volume. this leaves us with: [root inode ] [ snapshot copy of inode] [clone copy of inode]. Now we can still write to our normal volume by writing to that snapshot reserve that we talked about, and what we do when we create a flex clone is associate a *new* snapshot reserve with the clone. The cool thing is initially the clone takes up no space. Ok, that’s a lie, it takes up a coupe kilobytes, but we’re talking storage systems here, a couple of kilobytes is nothing. This is really useful for things like backing up a database, or doing QA -theres a whole bunch of use cases, such as, oh, avoiding the unneccessary dataloss of using snap restore in the event of a virus outbreak.

the actual solution

Ok, now that we’ve brushed up on our netapp basics, lets review our problem:

• we’ve taken hourly snapshots
• our active file system is infected with a virus, we’ll assume the volume name is vol1
• so is our previous hourly snapshot and probably the one before that
• the normal recommeded solution would be to snap restore to hourly.2 (three hours previous) before the virus broke out

Using this method would mean that we would lose three hours worth of data. That’s not good. Preventing data loss is the whole job of a storage admin, as far as I’m concerned. So, an alternate method:

1. take cifs offline, using the cifs terminate command
2. on your admin host copy the file /vol/vol0/etc/cifsconfig_share.cfg -> cifsconfig_share.cfg.YYMMDD.pre-virus.bak
3. make a clone of vol1 called “vol1_clone” with the command: vol clone create vol1_clone -b vol1 hourly.3
4. open /vol/vol0/etc/cifsconfig_share.cfg and do a find and replace, changing every instance of vol1 to vol1_clone
5. run the command cifs restart
6. run the command cifs shares -add oldvol1 /vol/vol1
7. lock it down with cifs access <your user> “Full Control”

So what did this do?

We quarantined the file system by stopping cifs and removing client access. Then we created a new clone of the volume that had been infected off of the snapshot we new was clean. We updated all of our shares using a quick find and replace giving our users access to their data, so they can get back to work. We also exposed our old volume via a new cifs share, which we’ve restricted only to our user, who theoretically knows better than to muck with infected files. So why did we do all this work? Later, when new virus definitions come out, we can scan that volume over night and have our anti-virus program go clean up our mucky infected files. Once we do, we can open up the share, send out an email, and give our users the ability to recover any important documents that had been created during those three hours, saving extra work, and potentially saving compliance headaches.

There is a final part of this, in which we split off the clone using the vol split command. I would note – you will (at least temporarally) need to have enough free space on your aggregate to contain both of the volumes, w/o any space savings. Once your split is finished, you can do a vol offline vol1 ; vol destroy vol1 to get rid of the old volume and free up that space.

I think this is a better solution to the problem, and one that’s much more elegant than using a snap restore. I’d love to hear any feedback or improvments to this process, so if you found this useful, or can think of a better way, please let me know.