AACS processing key, and the C&D letter to come

May 1st, 2007

I’m doing my part in adding yet another place where you can find the AACS HD-DVD processing key. I’m a firm believer in fair use, and the DRM scheme implemented by AACS clearly limits my abilities to use content that was legally purchased. I buy all my music on iTunes, I buy all my movies from various legal sources, and I get my TV from either iTunes or DirecTV. I’m in no way a pirate, and the use of the below key does not constitute piracy. While it might be used for illegal purposes, so can a gun, or so can pretty much any device. Don’t punish the individuals who posses the “weapon”, punish the individuals who commit the crime.

AACS processing key (09 F9 11 02 9D 74 E3 5B D8 41 56 C5 63 56 88 C0)

Posted in uncategorized | 1 Comment »

Review of Penn and Teller: Bullshit! Immigration

April 26th, 2007

Penn and Teller did it again, this was a great episode. I’m glad to see them taking on the immigration issue. Most American’s are opposed to immigration, and support stricter border controls. I think changing that opinion will be a hard pill for most American’s to swallow.

Penn and Teller are critical of the government’s plans to install 700 miles of fencing on the border at the cost of $60 billion dollars. They setup a mock version of the wall and show how easy it is to circumvent. Their illegals are able to get past it in around 3 minutes using a variety of different techniques. In addition, they pointed out that majority of illegals actual enter the country on legal visas and overstay their welcome. I think the most fascinating fact that they pointed out is that the contractor, golden sate fence company, who was hired to build the wall was actually fined for using illegal workers!

The one thing that they didn’t get into that disappointed me a little bit is the philosophical side of immigration. Immigrants are the core work force of this country. America was built on a tradition of immigration. By limiting immigration, you’re increasing manual labor costs, which in turn increases prices and lowers the value of the dollar. Immigration laws are another great example of “unintended consequences”. On the face it sounds like a good idea, limit the number of people entering the country, thereby ensuring jobs for the people already here, lowering the demand on social services, and controlling population growth. The problem however is that if you try and control the immigration you are forcing the “illegals” deeper into a black market. Mexicans will always find a way to get here, there is no physical way to stop them. Once here, the harder you make it for them to work, the harder you make it on everyone. You not only put the “illegals” into harms way, but you put Americans into danger as well. If their existence here is criminal, they will avoid authority at all costs. They might be witness to a crime, but they won’t report it. They might be in medical danger, but they won’t go to a hospital. They might resort to identify theft in order to remain here. You’re forcing honest, hard working, principled individuals to become criminals, just so they can provide for their families and better themselves.

A bit closer to home, I have a guy that takes care of my lawn every summer, let’s call him Bob. He is one of the hardest working guys I’ve met. He does a fantastic job, is punctual, and cheap. Of course, he is also probably illegal. Last fall he came to me and said “I’ve got to go back to Mexico, so I can’t take care of your yard next year”. Of course I don’t know the details, but I can only imagine. Why would we do this to someone who wants to work hard and make money. He has capitalistic intentions, he knows that he has to earn what he makes. He embodies the spirit and work ethic that this country was founded on. I’m proud to pay him to do the work, I feel a sense of satisfaction in our transactions. It’s hard to explain, but he lives for the ideas and principals that I believe in with all my heart. Yet someone has the balls to tell him that he can’t work here? That what he is doing is illegal.

Before we turn away men like Bob we need look in our own backyard. Some American’s are nothing more than looters, living off the system, living off our tax money, our social services, and our charity. Perhaps we should worry more about our existing problems in society before we cast stones on the immigrants. Instead of our money being taken at the point of a gun by our government to feed these American looters, maybe we should embrace the work ethic of our neighbors to the south and force the unproductive members of our society to take these men as examples. I say open borders for all, any man that has a work ethic like Bob and can help make this country a better place is welcome here in my book.

Here is the Libertarian party’s official stance on immigration.

The Issue: Our borders are currently neither open, closed, nor secure. This situation restricts the labor pool, encouraging employers to hire undocumented workers, while leaving those workers neither subject to nor protected by the law. A completely open border allows foreign criminals, carriers of communicable diseases, terrorists and other potential threats to enter the country unchecked. Pandering politicians guarantee access to public services for undocumented aliens, to the detriment of those who would enter to work productively, and increasing the burden on taxpayers.

The Principle: The legitimate function and obligation of government to protect the lives, rights and property of its citizens, requires awareness of and control over the entry into our country of foreign nationals who pose a threat to security, health or property. Political freedom and escape from tyranny demands that individuals not be unreasonably constrained by government in the crossing of political boundaries. Economic freedom demands the unrestricted movement of human as well as financial capital across national borders.

Solutions: Borders will be secure, with free entry to those who have demonstrated compliance with certain requirements. The terms and conditions of entry into the United States must be simple and clearly spelled out. Documenting the entry of individuals must be restricted to screening for criminal background and threats to public health and national security. It is the obligation of the prospective immigrant to demonstrate compliance with these requirements. Once effective immigration policies are in place, general amnesties will no longer be necessary.

Transitional Action: Ensure immigration requirements include only appropriate documentation, screening for criminal background and threats to public health and national security. Simplifying the immigration process and redeployment of surveillance technology to focus on the borders will encourage the use of regular and monitored entry points, thus preventing trespass and saving lives. End federal requirements that benefits and services be provided to those in the country illegally. Repeal all measures that punish employers for hiring undocumented workers. Repeal all immigration quotas.

Tags: , , ,
Posted in Uncategorized | 12 Comments »

Gun control in America

April 24th, 2007

I’m sure you could take a guess and figure out that I am an atheist, a libertarian, and generally considered a nut job by my peers. I find it interesting how seemingly separate areas of ideology (religion and politics) so often intersect when you become a free thinker. Almost every libertarian I know is an atheist, and almost every atheist I know has at least a slight libertarian bent. I mentioned in an earlier post that I had been reading Atlas Shrugged by Ayn Rand. I just finished the “John Galt Speaks” chapter, which is essentially a 58 page monologue describing Ayn Rand’s theory of Objectivism. It amazed me that this book, and this philosophy which was written over 50 years ago, describes exactly what I believe today. It amazed me that Ayn Rand was essentially an atheist libertarian, before it was socially accepted to be so (Not that it really is today either I guess). That isn’t the point of this post though.

One of the core tenants in Ayn Rand’s objectivism philosophy is in relation to use of force. The use of force should be reserved only for occasions where force has been used upon you. If your life is threatened by a gun, you should use a gun to defend you life. The problem is that, our society has taken it upon itself to limit the ability to defend ourselves when we are threatened by force.

In the wake of the Virginia Tech shootings, everyone seems to be shouting about new gun control. The re-introduction of the federal assault weapons ban has new momentum thanks to Carolyn McCarthy touring the talk shows and news programs speaking as if she was an authority on gun control, even though that bill would have had no impact on the shootings that took place. A new law has been introduced to close certain “loopholes” in the NCIC system for background checks. The new law would force doctors and other mental health professionals to hand over confidential mental health information to the government, in violation of our privacy. All of this is a vain attempt to tackle the problem by punishing the victims, and enabling the perpetrator.

Of course it is already illegal, even for concealed carry permit holders, to take a gun onto a college campus, but that didn’t stop Cho from doing it. The harder you make it to legally obtain a gun and to carry that gun in self-defense, the harder you are making it for victims to protect themselves. If just one of the staff members or students at Virginia Tech had been legally carrying a firearm, how many victims would there have been? I’m quite certain that it would have been less than the 32 that were killed, and 15 that were injured. It’s interesting to note that of the 14,000 concealed carry licenses issued in Oregon, only 4 individuals (0.03%) were convicted of criminal (though not necessarily violent) use or possession of a firearm. That rate is lower than the crime rate among police officers! The point is to say that carrying a gun does not make you a criminal, and the vast majority of those who do carry a gun do so in a safe and responsible manner.

As a society we need to wake up and realize that more laws, more protections, more limitations of our freedoms are never the answer. Our laws might not have caused Virginia Tech, but they enabled the perpetrator to take advantage of those of us who abide by the laws. Cho was able to kill far more individuals than he would have had these controls not existed. You can’t assume that someone who is capable of breaking the law and committing a crime like murder is going to be deterred by a lesser law like gun control or the inability to carry a weapon on campus.

This week, in response to the shootings, and out of fear of what kinds of laws might be passed in their wake, I’ve taken several steps to protect myself and my family. I’ve filed the required paper to obtain my concealed weapons (CCW) permit. I’ve purchased an additional sub-compact Glock 26 that will be my primary weapon for concealment, and I’ve purchased a AR-15 rifle in fear that it will soon become illegal to obtain. None of these are things that I would have considered doing a few weeks ago. Now, I feel compelled to do it in order to protect and exercise my 2nd amendment right.

jrvglock.jpg

Tags: , , ,
Posted in Uncategorized | 14 Comments »

10 steps to organize your digital life on OS X and Windows

April 10th, 2007

In meat space my wife and I are far from organized. It’s a real problem because we are always losing important things like our car keys. Unfortunately, this how-to won’t help me solve that problem. In the digital world however, I’m extremely anal retentive and organized. Everything is in its place and easily searchable. It allows me to be much more productive since I don’t have to waste my time trying to find things. I’m using a Mac and OS X Tiger at home and Windows XP in my day job at work, so that even further complicates my digital lifestyle. A lot of these tips can be platform agnostic and work under Windows and Linux as well. These suggestions work hand in hand with an indexed search capability. In OS X we have the built in, and fantastic, Spotlight. On Windows XP you can download Google Desktop. Of course Windows Vista has search built in. On Linux, if you are using Gnome there is Tracker. Also on OS X, take a look at Quicksilver

In no particular order my tips are:

1) Directory structures: In your personal document space (Your home directory on a Mac or Linux, “My Documents” on Windows) you need to create a document hierarchy to store your files. Depending on the amount of documents that you store, your system might include date as a component of the hierarchy so that your folders don’t become too large and unmanageable. I first divide my home directory up with a folder for each of the categories of documents that I might have. Word processing, spreadsheet, presentation, and notes. Under that I have it broken down by type. These are categories of the types of projects I might be working on. At work, this is the name of which of my clients for whom the document is relevant. From there, I have sub-directories for each year. 2007, 2006, and so on.

2) File names: All of my files are named with a standard convention. The convention is PROJECT/CLIENT_DESCRIPTION_YYYYMMDD_TYPE.EXT. Including the document type is helpful because sometimes the file extension doesn’t always give it away, or on some machines file extensions might be hidden. So for example if I had a Excel document for a client named Acme that contained a data extract, I would name the file ACME_DATAEXTRACT_20070410_EXCEL.xls

3) Music: I use iTunes to manage my music library. Part by force, since I own 4 iPods and an Apple TV, but part because I enjoy it. I think iTunes is a great interface for music management. I’ve done a few extra things in iTunes that help me stay organized. One of the most important data fields on a music track I find is “Album Artist”. This field is a recent addition to iTunes, but it really makes finding my music easier, especially from the iPod interface. For the music you purchase from the iTunes store, this is typically done for you already. If you’ve ripped your CDs, the CDDB database doesn’t always have this information however. The first thing I do after I’ve ripped a new CD is go and verify if the Album Artist field has info, and that it is correct. If not I manualy update this field based on the artist. This is especially useful on CDs that have a large number of collaborations on them. The second iTunes tip I have is use the rating system. When you are listening to your music, take the few seconds to go and rate the song. If you have a large music library, it is really handy to be able to just put on a playlist of your top rated songs when friends are over. You don’t want them to know you have the Macarena in your library!

4) Feeds: I use Google Reader to organize my feeds. In Google Reader you are able to setup multiple folders that each can contain multiple feeds. I have a folder for friends blogs, a folder for tech news, a folder for blog research, and a folder for each of my other interests. I can quickly see where new articles are and view just the topics I’m interesting in looking at. I also setup keyword searches on Google Blog Search and Google News and add the feed into Google Reader. This lets me get all the information I want, in a single easy to digest location.

5) Photos: We take a lot of digital photos, and we like to publish them online so that family can view them at their leisure. We have literally thousands of digital pictures that we have taken over the years. Keeping them all organized is no easy task. Recently I’ve begun the painstaking process of tagging all of the pictures in iPhoto. I tag the photos based on who is in the picture, the location of the picture, and anything that is particularly unique to the picture. Once the photos are tagged I upload them to our gallery using the iPhotoToGallery plugin. The gallery itself is powered by Gallery, a fantastic open source PHP based web gallery.

6) Address book: Now my address book organization is a bit tricky. As I mentioned above, I live both in the Mac and the Windows universe when it comes to my day job. At work we use Exchange and I carry around a Blackberry that is synced to Exchange. At home, I use my Mac, and I soon hope to have an iPhone. It’s a real challenge to keep my contacts synced between all these devices without spending too much time on it. In AddressBook 4.0 there is a handy feature that allows Exchange users to sync their Mac address book with Outlook Web Edition. Unfortunately for me, my company is still on the very old Exchange 5.5, and so I don’t have outlook for the web yet. While I openly admit this solution is far from perfect, it’s the best I have been able to come up with. Currently, I use my Blackberry as the transport medium for contacts. Exchange and my Blackberry work very well together, so I can always trust them. The problem is getting that same information onto my Mac. PocketMac makes a great utility that syncs your Blackberry with iLife. If I set this up as a one way sync (Blackberry to Mac) it works fairly reliable. The downside is that in order to add a new contact I have to do it with either my Windows PC or my Blackberry. I hope a better solution presents itself soon, or perhaps I might just dump my exchange dependancy all together when the iPhone arrives.

7) Chat contacts: Like most Mac users, I use Adium for my instant messaging. Adium is great because I can merge contacts from different networks into a single entry on my contact list and then set the network priority so I can choose which protocol to use if available. If you have the same person listed twice in Adium under your contact list, drag the contact from the lower priority network on top of the contact for the higher priority network. Adium will then ask you if you want to merge these contacts.
Another handy feature of Adium is having contact information pulled from your Address Book. This way their display name is listed as their real name instead of whatever catchy tag they chose to display as their alias. The last adium tip I have is how I organize my contact list. In Adium you can sort your contact list by status (under the view menu), and then choose to only view online contacts. This greatly reduces the number of contacts displayed and lets me quickly see who is available.

8 ) Email: Again, like Address Book email is a challenge in the hybrid Mac/Windows world I am in. Thankfully my email addresses are fairly separate. Work email stays on Exchange and personal email sits in Mail.app. Since I have my own domain name I use the fantastic Gmail for domains service. Google acts as my mail server and stores all my incoming messages. They are also kind enough to do spam filtering and provide an amazing ~16GB storage limit. I enable pop mail on the account and then fetch all my email down to Mail.app for local viewing and management. Once in Mail.app I use smart mailboxes to automatically sort out messages from the mailing lists I subscribe to. In Mail.app click the cog icon in the lower left, below your list of folders. From there, chose “New Smart Mailbox..”. With this interface you can specify under what criteria the mail message should be moved to the folder. For most mailing lists it is easy because typically a ID tag is prepended to the message subject. You can also use smart folders to create for individuals who email you a lot, so you can immediately see if you have new messages from them. That way you can give them your attention, or ignore them as the case may be.

9) Calendar: Ok, yet again I get bitten by the Exchange vs iLife issue. Similarly to how I manage my contacts I manage my calendar. I use the PocketMac software to import my calendar entries from Exchange/Blackberry into iCal. In iCal I have 2 separate calendars, work and personal. In the work calendar I have everything that is synced from the Blackberry, in the personal calendar I have everything that falls outside of work. This has worked out well because it allows me to stay organized but still keep my personal appointments completely private from my colleagues on our Exchange server. The only downside to this approach is that I do not receive reminders on my Blackberry for my personal calendar appointments. Again, hopefully the iPhone fixes this problem

10) Bookmarks: Now this is an area where the Windows/Mac problem is truly solved. I use del.icio.us to organize all my bookmarks. I can tag them so I know what they are about and I can easily retrieve them later. On Windows there is the new Firefox Extension and on my Mac and Safari I use Pukka combined with a simple javascript entry in my Bookmarks bar

javascript:document.location.href='pukka:url='+encodeURIComponent(location.href)
+'&title='+encodeURIComponent(document.title)+'&extended='+encodeURIComponent(window.getSelection());

When clicked, the above javascript sends your current URL to Pukka, which the gives you a prompt to tag it and upload it to Del.icio.us. The end results is that I have my bookmarks well organized and tagged on all of my computers. To access my bookmarks I just subscribe to the Del.icio.us RSS feed in Safari.

Can you think of any I might have missed? Have a solution to my Windows/Mac Address Book and Calendar problems? Drop a note in the comments.

Tags: , , ,
Posted in Uncategorized | 1 Comment »

Penn and Teller: Bullshit! Wal-Mart

April 5th, 2007

I have to admit, I’m a huge fan of Penn and Teller. When I heard that they were going to be covering Wal-Mart on an upcoming episode of Bullshit I was extremely excited. Like many small and medium size towns across the country, Wal-Mart hysteria has come to my home town of Longmont, Colorado and I was in the front lines of the debate.

Back in 2003 I had decided to run for City Council here in Longmont, and at that time Longmont was looking to annex a plot of land right outside of town by request of the Wal-Mart corporation. The plan was to build a new Super Wal-Mart store on the lot, and the city wanted to get the tax revenue. Typical to these decisions, the public outcry was immense. The anti Wal-Mart groups arranged speakers from all across the country to come and petition our City Council against the annexation. Of course, in the end cooler heads prevailed and the City Council voted to annex the land, and now we have a brand new Super Wal-Mart (to complement the smaller Wal-Mart across town).

Penn and Teller did an okay job in this episode, I’m just a bit disappointed that it didn’t go into more detail around the pure capitalistic nature of Wal-Mart, and really get down into the philosophy behind it. As anyone who knows me can attest, I’m a hardcore Libertarian. I live, eat, and breath free market capitalism. Nothing gets me more excited than seeing it in action, and nothing pisses me off more than special interest groups with agendas trying to stop it.

You see, Wal-Mart isn’t evil, capitalism isn’t evil, outsourcing and sweat shops arent’ even evil. They are all the natural product of a free market doing what it does best. Wal-Mart is one of the most amazing American success stories ever. A company that was able to find efficiencies in everything they do. Reinvest their profit into smart growth, cut costs where they can, negotiate bulk buying deals with manufacturers. There is a reason why Wal-Mart is the number one retailer in the United States, it is because they are the best at doing what they do, selling us household items at cheap prices. Wal-Mart isn’t here to put small business out of business. If that is a by-product of giving the consumer choice, then that is the essence of the free market.

What is evil is groups that preach the “social good”. Perhaps this is because I just finished reading it, but I’m reminded of Atlas Shrugged by Ayn Rand. The world is so concerned with the social welfare of the disadvantaged, the poor, the needy, that they “loot” from those who are productive members of society to give to the unfortunate. The same thing is happening with these anti Wal-Mart groups. They are going around spouting about the evils of Wal-Mart, using the plight of the common man as their evidence. Their motive may be well placed, but the outcome of such actions will eventually mean that Wal-Mart won’t be able to open new stores. They won’t want to deal with the headache and the legal loopholes that are put in place against them. Like the example Penn and Teller gave in Chicago requiring Wal-Mart to have a higher minimum wage than other retailers (Which gets me on an entire different minimum wage tangent, which I will save for a future post). The hurdles they will encounter will be so high that it will impact the profitability of a given store, and then they won’t have the justification to build it. What happens then? Who really suffers? Is it Wal-Mart, or is it the common man that now has to pay more for his television set at the local electronics store? Or is it the unemployed mother who still can’t find a job because nobody will hire her? Maybe it is the city who now has less sales tax because their residents are driving to the next town over to shop at Wal-Mart?

One of the great things about this country is that not only do we get to vote on election day. We get to vote every time we go to the store. Consumers vote with their dollars. They are saying that they want a higher quality of living for less money. They are saying that Wal-Mart offers them what they want at the best price. If the established businesses are adversely affected by the opening of a new Wal-Mart then that is the product of the kind of change the market is dictating. To survive, the established businesses must learn to adapt to the changing economy. People want convenience and they are casting their ballot loudly and clearly with every dollar they spend at these “big box” stores.

Tags: , , ,
Posted in Uncategorized | 30 Comments »

Outsourcing to India, the other perspective.

April 5th, 2007

I recently had the opportunity to visit Delhi on a business trip for a couple of weeks. It was an eye opening experience for me to say the least. It is shocking to see just how different the two cultures are. The extreme poverty next to the extreme wealth. It’s inspiring to see the opportunities in India that the globalization of the world has made available to them. The fact remains that just a few years ago this was considered a third world country, and you can see why. Much of the city has no running water or sewage. What water is available is contaminated with hundreds of different types of potentially deadly bacteria. What sewage is available is dumped straight into the Yamuna river, which coincidently happens to also be their only source of water. The air is full of heavy pollution, and you can’t see more than a few hundred yards away because of the dense haze. Your eyes and lungs burn from inhaling it, and the smells are overwhelming. The floors are all marble because things get so dirty, it’s nearly impossible to keep carpet clean. The drains are filled with moth-balls to keep bugs from crawling up them. There are scarce traffic controls, and where those traffic controls exist they are completely ignored. The streets are filled with cars, buses, trucks, motorcycles, auto-rickshaws. It would take some serious courage just to drive in Delhi. The hotel had a large nine foot wall around it, with armed guards. We had a private car and driver take us to the outsourcing facilities, where we were again surrounded by a nine foot wall with armed guards. On the outsides of these walls were beggars, looking to get money off the very affluent workers. For comparison, one of the software developers on our team just starting out makes around Rs 175,000 or $4,074.39 USD per year. This is the entry level salary and can go up quickly with experience. In Delhi, the richest city in the country, the average salary according to the Times of India is Rs 53,976 which is $1,256.68 USD (As of April 5, 2007), compared to the rest of the country which is Rs 23,241, or $541.10 USD.

Globalization and outsourcing are all over the news we read. Everyone has their opinion on how sending American jobs to foreign countries is bad for our economy. You can’t escape from this mindset. It’s drilled into us by the media, our friends, and our co-workers. Just yesterday I was confronted with the subject by a technician who was visiting my house to install a new Satellite TV dish. He cursed in frustration about the call center being located in India and complained about how the equipment he was installing was made in Mexico or China. What frustrates me when I hear these claims is how little information people have to back them up. They are just regurgitating broad generalized claims that they have previously heard, or worse, they are just speaking based on emotion instead of facts.

The truth of the matter is that outsourcing, be it call centers, software development, or manufacturing, all help the United States economy. We live in a capitalist country. Capitalism finds its roots in our currency, the mighty dollar. A dollar is used only as a representation of effort. If I work my 40 hour per week job and get paid my salary, my salary is just a representation of the amount of effort I gave my company, and in turn the value they placed on that effort and were willing to pay me. How I spend that money translates into the amount of effort it takes to produce a product, and what I am willing to pay for that effort. That makes the dollar our purest expression of effort, and therefore resource.

Now it is well established that outsourcing jobs to other countries is significantly cheaper than having them performed here in the United States, otherwise there would be no incentive to do it. The money, or resource, saved by having these jobs done overseas can be reinvested into our economy to make it stronger. A dollar saved is a dollar that can be spent elsewhere. The end result is that a company can do one of two things with the money saved. Either they reinvest it into the company or they lower the cost of their product to gain an edge against their competitors. In either case, again the United States economy is the beneficiary.

If the decision is to reinvest the money into the company, this typically translates into research and development. Americans have always been good at innovating, and it’s a perpetual cycle that as we innovate we create new work for ourselves to do. These jobs, because of their complexity in nature, are things that can’t be easily outsourced, and so the cycle continues.

If the decision is to lower the cost of your product, then that means the value of our dollar has increased. We can now purchase something with fewer dollars than it would have previously taken. Thus, a dollar saved is a dollar that can be spent elsewhere by consumers. If you are a business, you go back to step one and decide how you are going to use this money you just saved because of the lower purchase price. Lower the cost of your own product, or reinvest in innovation. If you are a consumer, your quality of living just went up because now you can get more with the same amount of money that you previously had.

If I asked you fifty years ago what a software developer was, you wouldn’t have heard of such a thing. Now as the efficiencies of software development are solved, we have the ability to move some of these jobs overseas to reduce cost and save those American dollars. In fifty years what jobs will be common place enough to justify outsourcing? Perhaps we might be outsourcing our “genetics engineers”.

My personal experience with outsourcing comes in the software development area. There are a lot of American companies who have found great cost savings in doing simple software development “off-shore”. When I say simple software development I mean mundane things. In our case, that is the implementation of our customers’ business systems on a pre-built platform. I used to be a software developer for the company I work for, then several years ago our development jobs were outsourced to India. Instead of losing my job, I moved into a project manager role and now supervise the efforts of our off-shore development team. During my time working with our Indian developers, I have learned a lot about the differences in our work ethic, our culture, and our abilities.

I’m often asked, “What is the most difficult thing you have had to deal with in terms of outsourcing?”, and the answer might surprise you. The problem we have with our developers is not their ability, not their language (India is the largest English speaking country in the world), not even the time-zone differences (which do cause us a number of problems). The biggest issue we have is the fact that they are generally not “out of the box” thinkers, or if they are, they aren’t capable of communicating their ideas to us. They prefer to be treated in a very literal fashion. They need to be told exactly what to do, and how to do it. They rarely offer suggestions as to how changes in process could improve their performance. They need to be told exactly how the code should work, what it should do, and how to test to make sure it does what it is supposed to do. This goes back to my earlier point about innovation. The literal attitude, the cultural differences, and their lack of understanding our “business reasons”, makes it very hard to communicate their original ideas to us. I don’t want to undermine their ability, they are very skilled. They know and do their jobs with great competence, as long as you are telling them what it is they need to be doing. Several of the members on our team over there even have their PhD in computer science. They can’t get much more skilled.

The other difference that was hard to overcome initially is working styles. When we first started using the offshore team, they were very impersonal. There was a clear hierarchy in India that was followed, and we were only able to communicate with them at certain levels of the hierarchy. It created a “black box” of resource for us that was very difficult to manage. We had no direct contact with the individual contributors on our team. We were left in the dark quite a bit about progress on a given project, questions they might have, and issues they might be facing. We handed over a specification to the black box, we were given back a “delivery date” and then we sat around waiting until our delivery showed up. The delivery was ALWAYS on the day it was promised, never early, never late. The delivery took much longer than if we had done the same project on-shore, and was of poorer quality. After months and months of discussions, and changing the process around to better fit our “American” working style, we were able to open up the lines of communication and establish the direct contact with our developers and encourage them to ask us questions when they needed direction. We’ve been able to establish with them that delivery dates are meant to be targets, that if you get the work done early, it is ok to delivery it early. Conversely if the project is taking longer than expected, sometimes it is ok to be late. We discovered that our team had been working long overtime hours on some at risk projects in order to try and meet promised delivery dates, which in turn significantly impacted quality. Thankfully those days are behind us.



taj.jpg

This picture of a couple members of our team, my boss, and I is a tribute to this mentality. When we first went offshore, we asked them for a team picture so we could put faces to the names of those we were working with. They sent us a photo, with everyone in the team looking very stern with their arms crossed for the camera. We got a kick out of this, but in reality it underscores the cultural differences. At this point in our relationship with them, they were unable to be friendly and open with us, and were therefore very reserved. When we were over in India we decided that we would all pose in the same manner as sort of an inside joke to show how the team has changed.

Initially we had very high turnover on our team. Again, looking at it from a black box model, the management in India was quick to replace one member of our team with another, assuming it would be transparent to us and we wouldn’t mind. This contributed to the above problems because we were then constantly building relationships from scratch, and re-training them on our processes and technology. Thankfully we were able to solve some of these problems and ended up with a highly competent team, which has had very minimal turnover. We now have a group of guys that we would call our friends. We’ve had some of them over for visits here to our office in the United States, and I’ve had the privilege of spending time with them in India.

After we got past the initial intimidation that they seemed to have towards us, they really opened up. They are super nice, warm, friendly, caring individuals who really enjoy what they do. They of course all have dreams of getting an H1B visa and moving to the United States at first opportunity. They participate in the lottery ever year in hopes that they might be selected. It is a shame that our immigration policy is the way it is, and that we don’t have truly open borders. These guys deserve the opportunities that America has to offer more than a lot of its own citizens. A man should be judged not on his country of origin, his race, his language, but on his ability to produce. These guys are good at what they do, and are hard workers. They can produce, which is what this country is all about. An individual’s ability to produce.

In summary, globalization is good for all who are involved. We are creating opportunities for those who previously had none, and we are saving American’s money so that we can continue to innovate. We can continue to be the global leader in technology. There will always be small scale negative impacts of globalization, when a group of individuals lose their jobs to outsourcing. This is the price that we have to pay in an every changing society. They will have to adapt and find new jobs, jobs that can make better use of their skill sets. I don’t have any fear of globalization, I encourage it and look forward to living in a truly global society.

There are a few books on Amazon that I recommend reading on this topic. These are:

The World is Flat – By Thomas Friedman

The Lexus and the Olive Tree – By Thomas Friedman

Tags: , ,
Posted in Uncategorized | 1 Comment »

What is Web 3.0? A review of the ICWSM

March 27th, 2007

What is Web 3.0?

Just when you started to get used to the idea of Web 2.0, the trend setters on the web start tossing around the term Web 3.0. Can’t they just let us get used to one version before pushing another one on us? Fortunately Web 3.0 is in its infancy, and nobody is certain of what it really means at this point. The direction of the internet and its users is a fickle thing. It’s hard to predict what they are going to want and use. The next big thing is all about the right idea, in the right place, at the right time.

I’ve been attending the International Conference on Weblogs and Social Media conference the last couple days and listening to some of these visionaries talk. While there certainly is no consensus about what the future holds, there are definitely some key trends to take note of. I decided to go this year because it just so happened to be local for me. Normally I wouldn’t have taken the time off of work, incurred the travel expenses, etc just to go to a conference like this. For some bizarre twist of fate they decided to hold the conference in my home town. Given my personal work in blog research, it seemed like a no brainer to go to this conference.

The topics presented at the conference were fairly diverse, ranging from sentiment analysis to gender bias in blogs. There were three invited speakers and a number of academics presenting published papers. I have to say that by far I got more out of the invited speakers than the papers, although there were a few the sparked my interest. The three invited speakers were Danah Boyd (a prominent blogger), Andrew Tomkins (a researcher at Yahoo!), and Evan Williams (founder of Twitter). Here are some of the highlights from my notes.

Danah Boyd – “MySpace is *my* space”

In the talk she gave us a brief history of social networking starting with Friendster and went on to discuss a number of specifics about MySpace.

  • 93% of American teenagers have access to the Internet.
  • 55% of online American teens 12-18 have a profile on a social network site (That are willing to admit it in front of their parents).
  • 91% use it to talk to friends while very few use it to talk to strangers.
  • The number one spot in the “top 8″ is dependent on culture. Some cultures dictate that it should be your significant other, some your best same sex friend, and others say family members like your cousins.
  • 66% of teenagers use MySpace in a private fashion in order to avoid marketeers and adults (parents).
  • Teenagers think something is fishy on a page without ads. They suspect at some point they will be asked to pay for something if they aren’t being force fed advertisement.
  • Gadgetry has broken the gender barrier. More teen girls have game consoles, iPods, and trendy cell phones now.
  • If parents are flipping out, they know it’s going to be fun!

In addition to the key points about MySpace, Danah also discussed the direction that she sees social networking going. The future to her is about mobile devices. We already see the prevalence of SMS in Asian cultures, and it is slowly becoming a phenomenon here in the US with the introduction of sites like Twitter. The problem is how this move will take place given the current atmosphere in the mobile market. The major cell phone carriers control every piece of software on their phones and every byte of data that goes over their network. There is no self-interest for them to open these platforms and networks up for development. They want to be in control so they can utilize the technology to make as much money as possible. Because of the competitive nature, there is not likely to be any major cooperation between the carriers, which will stunt social networking’s growth.

Andrew Tomkins – “Social Media, Storage, and Data Analysis”

Andrew is a PhD member of the Yahoo! Research team. His talk focused on several different topics, including the evolution of search, and Flickr.

Interesting things being done with search:

  • Moving away from the “10 results” model and devoting more real estate to targeted results.
  • Search is adding in shortcuts to specific verticals. For example, integrating weather and movie times right into the search results.
  • Google Co-Op allows content providers to integrate their results in your Google search.
  • The current approach that is used is naive, based mostly on regular expressions and filtering for target words.


Is search solved?

  • Search is really good at finding content on relatively open and static pages. Search does not integrate results where content might be buried in forums or other social network sites.
  • The data sits behind walled gardens and is generally unavailable. There is lots of money still to be made in search, but it will require a large capital investment and structured deals with the owners of the content to let search inside their walls.
  • For example, the Yahoo! Answers data is fully crawlable but the several billion posts in Yahoo! Groups are locked up from the outside.

Andrew also showed off a demo off a timeline based view of Flickr tags. This can be seen over at Taglines. He also mentioned that they are developing a “game” that users can “play” that will actually generate metadata and tags for Flickr images.

Evan Williams – “The Evolution of a Social Media Platform: Twitter “

Evan is the founder of Blogger, Odeo, and Twitter. He is one of the leading entrepreneurs in social media and brings a unique perspective to the conference. Most of his presentation talked about Twitter and the things they are doing. Twitter started just 9 months ago, and went through a complete revamp in November of 2006. Over the last few months the number of “tweets” being sent into Twitter has doubled month over month.

  • 2/3rds of the inbound tweets come from the web and IM, the remaining 1/3rd come from SMS
  • They are currently working on an API that will soon be released
  • While they don’t have hard data, their informal research indicates that the core demographic of Twitter is 30 year old web geeks, not teenagers as you might expect.

Evan also presented a quote from Lisa Reichelt about ambient intimacy that describes Twitter.

Ambient intimacy is about being able to keep in touch with people with a level of regularity and intimacy that you wouldn’t usually have access to, because time and space conspire to make it impossible. Flickr lets me see what friends are eating for lunch, how they’ve redecorated their bedroom, their latest haircut. Twitter tells me when they’re hungry, what technology is currently frustrating them, who they’re having drinks with tonight.

There was also a quick demo of Twittervision. This is a site developed by a 3rd party that integrates Google maps with Twitter. The result is that users who send a location code with their tweet (l:Boulder, CO for example) will be shown on the map display.

Summary

While the future is far from certain, one thing is for sure. The future of social media and “web 3.0″ will be focused around two key areas. Mobility and search. Developers needs to come up with better ways to get at the information you need and make it simple to do from mobile devices. While some predict the death of sites like Twitter, I think they are ground-breakers in their field. When blogging can become a commodity that is approachable to anyone, and all of that data is well organized and searchable, that is when we can say that web 3.0 has arrived.

Tags: , , ,
Posted in Uncategorized | 5 Comments »

Optimizing Wordpress and LAMP to survive the Digg effect

March 26th, 2007

Anyone who has used digg for any amount of time has certainly come across a dead link. Invariably you click on the comment link hoping someone has posted a mirror of the content. You head over to duggmirror or coral cache to view the content. During the peak time right after an article hits the digg homepage, a given site might receive as many as 100 page loads per minute. If this page is hosted on Wordpress, or other MySQL based blog platform, combined with shared hosting, it’s a recipe for a crash. Your site will go down in flames and you won’t have a chance of recovering until the load goes back to a lower level.

Recently I had an article hit the front page of Digg. I was lucky because I had already done a number of things to optimize my server so that I survived. My page never went down, and was very responsive for the entire duration of getting dugg. My site is hosted on Wordpress on a server that I own and manage. The server hosts about 15 different domains, none of which are exceptionally high traffic sites. The server isn’t anything special either. It’s a single processor P4 3.0GHz with 512MB of memory running CentOS 4.4, Apache 2.0, PHP5, MySQL4. I do have webalizer running on my Apache logs and I have the output of those posted at http://vallery.net/stats/ if you would like to see what the Digg effect can do.

I never exceeded more than a 0.15 load average during the Digg. Load average is a computation of how maxed out a Unix based system is. You multiply the number of processors by 1, and this indicates the maximum load average that you can reasonably sustain. If the load average exceeds this number, than your system is over utilized. In my case, my max server load is 1 given that this is a single processor system. This means I was only using 15% of my system resources during the maximum load that was generated from the Digg.

In order to optimize my system I have done the following things:

1) I’ve installed the Wordpress 2.0 plugin wp-cache. This is something that every Wordpress blog should have installed. Especially on a shared hosting environment, it will dramatically increase your ability to handle high traffic. The plugin generates the HTML for a given page and then saves it in a cache file. When someone accesses your page, instead of fetching the content from the database it uses the already generated HTML in the cache to send to the browser. This eliminates a number of fetches to the DB and dramatically speeds up page load times.

2) Optimize Apache/MySQL to handle the expected number of database queries. The most common error you receive when viewing a Wordpress page that has been owned by the Digg effect is a database connection failed, or timeout. When connecting to a MySQL database you can either have a persistent connection, or generate a new connection for each request. There are a number of schools of thought as to which method is better, but generally speaking using persistent connections utilize more memory, so for me using a new connection for each request makes more sense. A lot of the PHP blog templates including Wordpress use these non-persistent connections as well. The rub comes in the maximum number of said connections. MySQL can be configured in your my.cnf file to say the maximum number of connections that can be created to the database. If each instance of Apache (which in turn represents concurrent vistors to your page) has it’s own connection, then you can quickly exceed this maximum number of connections. Apache similarly has a configuration option that dictates the maximum number of threads that can be concurrently running. This number is specified in the httpd.conf file. If the Apache number is larger than the MySQL number (which is the case in most default configurations) when you have more Apache threads running than MySQL is capable of handling, you get our nice “maximum number of connections” error message.

In /etc/my.cnf you want to set the variable “max_connections”. I recommend something reasonably high like 250.
In your httpd.conf file for Apache you want to set the variable “MaxClients”. This should typically be the same number that you selected for MySQL.

3) Set reasonable Apache timeouts. This means that no individual thread/connection can monopolize the system bringing all the other queries to its knees. This protects you from rogue “edge” cases.

In your httpd.conf file set “Timeout” to a low number like 30 (measured in seconds) and “KeepAliveTimeout” to something like 3.

4) If your site makes it on to Digg, use real-time monitoring tools to measure your server’s health. There are a number of command line tools available to help in this regard. The first I would recommend is the tried and true “top” command. This will display the processes running on your machine along with their associated memory and CPU usage. Keep an eye and make sure things aren’t getting out of hand. The second tool that I use is called “Tcptrack“. Tcptrack will need to be installed on your machine, but once it is it will give you a real time view of your incoming connections and bandwidth usage.

Happy Digging!

Tags: , ,
Posted in Uncategorized | 25 Comments »

Scalable story promotion

March 26th, 2007

I had some thoughts on the idea of scalable story promotion for the open source Pligg system. I thought I would share them here

I’ve been thinking about the best way to handle promoting a story/article from a queued status onto the main page and I’ve had a few thoughts I wanted to share with everyone.

The current scheme for promotion is very simple. Number of votes passing a defined threshold in the config file (and the story is fresher than X days). While this will work for very low volume sites, it doesn’t exactly scale well. In a site with a large number of users, and one would assume proportionately a large number of votes, this breaks down.

Our ideal system would be able to quickly determine that a story is of high value and promote it to the homepage based on the frequency of votes as compared to a large sample population. Using some basic statistics we can determine if a story is an “above average performer” and promote it quickly.

In order to accomplish this we need to take into consideration several variables, including:

1) Number of stories submitted in a given time period
2) The average number of votes over a given time period for all “active” stories in the queue
3) The standard deviation of votes over a given time period for all “active” stories in the queue
4) The target number of stories to be promoted to the home page in a given time period

For the purposes of explaining my ideas I am going to define the time period as one day. We need to calculate and story the answers to some of these questions on a regular basis. Ideally we create a new DB table that stores this information for us to easy lookup. This way we can also track information statistically and show trends. For my plans I plan on implementing a cron job that runs daily that calculates the required and stores them as follows.

1) Calculating the number of stories submitted is trivial. The SQL query I am using is:

select count(*) from links where link_status = ‘queued’ and date(link_date) = ‘2006-04-16′;

2&3) Calculating the average and standard deviation can be done with the following query:

select avg(link_votes) as ‘average’, stddev(link_votes) as ’stdev’ from links where link_status = ‘queued’ and date(link_date) = ‘2006-04-16′;

4) This can be set in the config.php file and up to the site administrator.

These values will be used to “predict” the future for our new stories. Each story will have a new variable that stores a floating point number. This number is the number of standard deviations above or below the mean (average). We need an additional check running on a much more frequent interval (I plan on using every 5 minutes) to update the items in the database with their new “score” and promote them once they pass a specific threshold.

I plan on calculating this as follows:

$score = //Z score that indicated if a story is above/below the mean
$numofvotes = //The number of votes that a given story has received
$stddev = //Standard deviation for the given time period as calculated above
$average = //Average number of votes for the given time period as calculated above
$numberofstories = //The number of stories submitted in the given time period as calculated above
$desirednumofstories = //Setting from config.php

The cut-off value determines a “score” threshold, or essentially a percentile that a story must fall into in order to be promoted. I plan on calculating this as follows:

$score = ($numofvotes – $average)/$stddev;
or in SQL

select link_id, ((link_votes – $avg)/$stddev) from links where link_status = ‘queued’ and date(link_date) = ‘2006-04-16′;

Once we have this score we need to decide if this story needs to be promoted or not. This is done by first calculating where in the rank order this story is likely to fall for the day using some basic probability statistics.

$rank = round($numofstories/(1+exp(-1.7*$score)),1);
Now we just check and see if that rank falls above our threshold and promote the story accordingly.

$desiredrank = ($numberofstories – $desirednumofstories)
if ($rank >= $desiredrank) then {
// Update story and set to promoted
}

This method should work well assuming that traffic is fairly stable from day to day. Since we are using the previous days data to predict the current days volumes if a sudden traffic spike is hit it will mean that a larger number of stories will be promoted than desired. This can be mitigated on larger volume websites by decreasing the “given period of time” from a day to something shorter. Additionally this whole idea could be retooled to calculate these variables on a per category level. Some sites might have much higher traffic, and therefore votes for one category than another.

I’m currently working on implementing the above for a site that I plan on launching in the near future, and I would love feedback, criticism before I do it.

Tags: ,
Posted in Uncategorized | 2 Comments »

Setting up an automated workflow to convert files for Apple TV on OS X

March 23rd, 2007

With the arrival of my Apple TV yesterday I needed a solution to get my Xvix/WMV/Divx files converted and imported into iTunes so that I can watch them. I already have Quicktime Pro, which with the recent release added the ability to “Export to Apple TV”. Since I have a lot of files, and no desire to sit around and convert these one at a time, it seemed like a perfect job for automator. I figured someone out there at some point had to have done something similar so I did a bit of google searching and found the required automator actions. Using the actions I found combined with the sample workflow they have already created for you, it is trivial to set-up a workflow that will convert to the Apple TV format and then import the file into your iTunes library. With a slight modification you can set it up a plug-in and attach it to a folder action. Now I have a simple drop folder on my desktop that launches quicktime pro and converts the file to an Apple TV viewable format, imports the file into iTunes, and cleans up after itself.

Here is a simple step-by-step guide to walk you through what I did:

1) Install xvid, divx, and wmv codecs.

These can be found here:

Divx
Xvid
WMV

2) Install the automator actions for compressing and importing into iTunes.

Download Quick Time Compression Actions and Workflow

3) Once installed you will have a directory on your desktop called “QuickTime Compression Workflow resources”. In this folder you will find a sample automator workflow called “Convert videos and add to iTunes”. Open this workflow in automator.

4) Delete the first step of the workflow which is “Ask for Finder Items”. Instead of being prompted for which items to convert, we want to setup a folder action that will automatically convert the files dropped in our folder.

5) Add a new first step to the workflow called “Get Selected Finder Items”. This action can be found under the “Finder” application.

6) Under the “Compress QuickTime Using Most Recent Settings” step change “Choose directory for converted files” to the desktop (or any other temporary folder you want to use).

7) Create a new folder on your desktop. This will be your drop folder, so call it something relevant. I called mine “Convert to AppleTV”.

8 ) Back in automator, click on file and choose “Save as plug-in”, choose “Folder Actions” from the “plug-in for” drop down. Give the plug-in the same name as your folder. Select your newly created folder for the “Attached to folder” option. Click save.

9) Since the script will convert whatever file you drop in your conversion folder using the last settings you used in QuickTime you’ll need to launch QuickTime with a test file and then choose “Export” from the file menu. Assuming you have the most recent version of QuickTime Pro you should have an option “Export Movie to Apple TV”.

10) That’s it!!! Now just close out of everything and drop your files into your new folder and watch as they are converted and imported to iTunes. It works great to leave your Mac on and then drop a bunch of files in the folder before you go to bed. When you get to your PC in the morning everything should be all ready to go.

To find out more about folder actions, check this page out:

Folder Actions

Tags: ,
Posted in Uncategorized | 56 Comments »

« Older Entries
Newer Entries »