Adam Caudill's Blog » Development

SOPA Is Inevitable

Adam Caudill — Sat, 07 Jan 2012 07:10:53 +0000

SOPA, while it’s not likely to be passed as-is, I would be willing to bet money that something SOPA-like will be passed. It may be watered down with many of the most offending parts removed, but for those backing SOPA it’ll still be a real victory. For them getting it on the books, even in a weakened form means it can be tweaked (and extended) later.

There’s been an amazing resistance to SOPA, from the boycott of GoDaddy to public statements from celebrities such as Adam Savage – the public outcry against this horrid piece of legislation has been quite inspiring. But how often will you be able to get so many people to stand up and take action before they start to lose interest? How many times can you raise the troops before the numbers start to dwindle; how long before the celebrities start fearing they’ll be branded in the media as extremist or crazy? How many times can you raise the call of breaking the internet and freedom of speech before the public gets bored and goes to read about the latest Hollywood divorce instead?

Here’s how I see it going:

Strip many of the worst parts of SOPA and get it through congress. By removing these offending pieces, those backing SOPA will try to make themselves look responsive to the community, and it’ll be played as a victory for the community in the media. All in all, if you aren’t paying attention it’ll look like a victory for the people.
Next year, introduce a bill to modify SOPA to change the wording here are there, edging it just a little closer to the original. If done carefully, it’ll be easy to dismiss those that try to stir up another outcry as over-reacting or even paranoid.
In a few years after a series of modifications, we have SOPA, just as broad and dangerous as originally intended – and the vast majority of people who fought SOPA would have no idea.

If you have a financial motivation to get something like this passed, they key to success would be patience. Chip away slowly at DMCA Safe Harbor protections, at what requires a judge instead of an administrative action, at transparency so that any action ends up happening behind closed doors. In enough time you’ve established a law that gives the US Federal Government a massive amount of control of the internet, without oversight – all in a way designed to get offending web sites off the internet as quickly as possible. To say it would be ripe for abuse would be a massive understatement.

Am I being paranoid? I honestly hope so – I really hope that there aren’t people out there looking to limit the freedoms we cherish for their own profit, but the fact that SOPA was introduced in the first place makes that hard to believe.

Masking Credit Cards for PCI

Adam Caudill — Thu, 20 Oct 2011 20:14:40 +0000

PCI DSS, the security standard for companies that handle credit cards, defines a number of rules as to how credit cards are handled. One of those rules, 3.3, is defined as follows:

Mask PAN when displayed (the first six and last four digits are the maximum number of digits to be displayed)

So based on this requirement I assumed that the code to do this would be common and widely available; much to my surprise there are rather few samples that do this, and of those I found they only showed the last four (which when you are handling a lot of credit cards, searching for an account by the last four isn’t all that helpful) and were often rather fragile.

So I whipped this up, hopefully it’ll be useful to others.

public static string MaskCreditCard(string value)
{
  const string PATTERN = @"\b(?:4[0-9]{12}(?:[0-9]{3})?|5[1-5][0-9]{14}|" +
    @"6(?:011|5[0-9][0-9])[0-9]{12}|3[47][0-9]{13}|3(?:0[0-5]|" +
    @"[68][0-9])[0-9]{11}|(?:2131|1800|35\d{3})\d{11})\b";
 
  var replace = Regex.Replace(value, PATTERN, new MatchEvaluator(match =>
  {
    var num = match.ToString();
    return num.Substring(0, 6) + new string('*', num.Length - 10) +
      num.Substring(num.Length - 4);
  }));
 
  return replace;
}

            view raw
            gistfile1.cs
            This Gist brought to you by GitHub.
          

The regex pattern is from Regular-Expressions.info and should detect most major cards.

Why Cringely is wrong about Java

Adam Caudill — Sat, 15 Oct 2011 19:47:52 +0000

A couple of days ago I was sent a link to Robert Cringely’s latest treatise: The second coming of Java – and to say I disagreed was a bit of an understatement. To me, it represents a fundamental flaw in his perception of developers, and more importantly the economics of software development.

The key to Cringely’s argument comes down to this:

When SSDs gain enough capacity there will be a shift from the Ruby world back to the Java world. Not for prototyping, because, well, it’s prototyping. But simply because the statement “Ruby is incredibly slow but I don’t care because my database is slower” will no longer be true.

What he’s missing here is the real reason people use frameworks like Rails; it’s not about it being Ruby, or being the latest cool thing – it’s about developer productivity. That’s it, and that’s all there is to it – Rails allows a developer to do more in less time. That’s one of the key reasons so many Java web developers jumped ship (though I can think of a few others), and what pushed Microsoft to invest so heavily in their MVC framework.

I could fully rehash the argument, but in what I consider to be one of Jeff Atwood’s best articles, Hardware is Cheap, Programmers are Expensive, he covers a key point to my argument – developer time is vastly more expensive than hardware. Atwood’s take on the issue is clear:

Clearly, hardware is cheap, and programmers are expensive. Whenever you’re provided an opportunity to leverage that imbalance, it would be incredibly foolish not to.

When there’s a choice between developer productivity, and spending money on hardware – the conclusion should be the same. It’s much cheaper to throw more hardware at a slower framework than it is to invest more developer time in a faster framework. For any non-trivial application, throwing more front-end servers at it will always be cheaper than slowing the development process down with a non-productivity-centric toolkit.

It’s simple economics; server hardware is getting faster and cheaper, developer time is only getting more expensive.

Pen-Testing Silverlight+RIA with SoapUI

Adam Caudill — Wed, 06 Jul 2011 21:39:37 +0000

I was recently given the task of ensuring that a Silverlight+RIA application that could contain private information was secure for deployment to a public web site. So I started searching for automated pen-testing tools that could work against Microsoft’s Binary SOAP protocol (msbin1, a.k.a “application/soap+msbin1“) and found only disappointment. For various reasons, it’s significantly more complex to pen-test a application using msbin1 than traditional SOAP + WSDL.

To properly test the services, I had to make a compromise: temporarily modify the application to expose a SOAP endpoint. While this changes the state of the application and thus reduces the validity of the tests, it does provide a reasonable way of testing the web services to ensure that they are behaving as intended.

The recently released SoapUI Pro 4 adds new security testing tools that makes this a viable (and attractive option). To get this working, there are a few small changes that need to be made to the solution:

First, you’ll need to add a reference to “Microsoft.ServiceModel.DomainServices.Hosting.EndPoints” which is part of the RIA Services Toolkit; this allows you to expose different End Points for the service such as SOAP and OData.

Next, you’ll want to add the following configSections entry to your Web.config:

...

Finally, to expose the SOAP end point:


 ...
 
  ...
  
   
    
   
  
  ...

Finally, just follow the instructions for SoapUI to setup your tests, and you can feel (just a little) more confident in your application. Passing with flying colors obviously doesn’t mean your application is bulletproof, but it helps to confirm that web service code is solid.

Now, while this does provide some insight into your application and should help find common issues, it’s not a replacement for a professional assessment by a qualified auditor. If you are handling credit cards or other highly targeted information, please consult a security specialist before a public deployment.

Happy 20th birthday Visual Basic!

Adam Caudill — Sat, 21 May 2011 05:50:19 +0000

Today I saw a post on Facebook by a friend of mine, Anthony Green, about writing his first blog post as a Microsoft employee (he has a personal blog as well, unfortunately he’s not written anything since 2008) – when I saw the title, I couldn’t believe it was 20 years already – seems just yesterday that I wrote about its 15th birthday:

Happy 20th Birthday Visual Basic!

My, what a journey it’s been. Almost fifteen years ago I randomly bought a copy of “Visual Basic 5: Deluxe Learning Edition” – I was just 15 at the time and wanted a new hobby, and writing software seemed like it would be fun. In those early days, I had no idea what career I would choose, and really didn’t intend for software development to become the dominant force in my life – I just wanted a better, more productive way to spend my time during the summer.

In the years that have went by, I became passionate about the field, and all it encompasses (possible obsessed, if you believe my wife) – it’s been the driving force in my life. Today, I manage a team of 6 developers, and have a fun start-up with some friends (that someday won’t cost me money every month) – and all because I bought that book. Overall, I have a lot to thank VB for, it really did get me started in this field.

Today though, my language of choice has moved on to newer options – I prefer bleeding technologies when I can use them – but VB will always have a place in my heart, and I’ll always follow its progress as it continues to transform and adapt to an ever-changing world. As the most popular .NET language (contrary to what many of the C# developers think), it plays a vital role in the development of the framework and the ecosystem.

In the conversations I’ve had with Anthony about the future of the language, I greatly look forward to writing about its 25th birthday; I expect those will be exciting times for the language and the entire .NET ecosystem.

VB6: Not so open source

Adam Caudill — Fri, 20 May 2011 06:33:23 +0000

Earlier today, a rather surprising tweet hit, being retweeted at least 80 times, including by a few rather influential people in the .NET world:

@RoyOsherove
Roy OsheroveMicrosoft announces to mvps at #msteched that VB6 will be released as open source on codeplex end of june! w00t
May 19 via TweetDeckFavorite RetweetReply

Needless to say, that’s not an announcement that anybody was expecting, but given the talk going on at the time – and the high-profile people talking about it, there wasn’t much reason to doubt. Announcing a product that has been dead for years is going open source would certainly be a strategy shift for Microsoft, but does it make any sense? Kevin Dente of Herding Code fame certainly thought that Microsoft had better things to release:

@kevindente
Kevin DenteInstead of VB6 I’d rather see MS open source IE6. Then at least we could build a standalone version of it.
May 19 via HalfwitFavorite RetweetReply

Shortly after the initial tweet, Doug Seven, the Director of Product Management, Visual Studio Tools & Languages, replied asking Roy Osherove (the original poster) to email him. Hmm, it’s starting to smell like something odd is going on. A couple of hours later, Doug set the story straight:

@dseven
dsevenThe rumors of VB6 going open source are simply not true. #msteched #vb6rumor #vb6
May 19 via webFavorite RetweetReply

To which, Roy then tried to unset:

@RoyOsherove
Roy OsheroveRT @dseven: The rumors of VB6 going open source are true. #msteched #vb6rumor #vb6
May 19 via TweetDeckFavorite RetweetReply

It’s worth pointing out that Roy Osherove currently has a full ten-times the followers that Doug Seven has, meaning his altered retweet was seen by more people (at least initially). For several hours word was going around, and accepted by a number of people who thought Microsoft was actually going to open the code to VB6 (including journalists who were too busy writing articles to do any fact checking) – all based on one person who made it all up.

Lesson here: be careful about what you re-tweet, it’s easy to endorse a lie as several people unwittingly did today (@blowdart summed it rather well).

You can’t fix stupid…

Adam Caudill — Tue, 10 May 2011 06:06:05 +0000

For those outside of the IT field, developers are looked at as miracle workers – through us, business leaders think anything is possible (and they often see no reason why we can’t work our latest miracle by the next morning). In reality though, we do work miracles; we save companies vast amounts of money every year through increased worker efficiency and automation, we enable new lines of business that wouldn’t be possible otherwise, and reduce energy costs because we keep the office lights turned off. Well, that’s more or less how they see us.

But for all of the wonders we are responsible for, there is one thing we can’t do (no matter what amazing powers some executives think we have to make them look better or earn them more bonuses):

You can’t fix stupid.

I’ve often described development as being a professional problem solver, and we are often tasked with rather challenging problems to solve. Sometimes the problems are purely technical – making something new possible that previously was impracticable or even impossible, sometimes it’s all about efficiency, other times it’s about image and controlling how people see a company. When the problem is the user though, you know you’re in for a rough day.

I was recently given such a task – the users weren’t listening to their supervisors and they wanted the software to force these users to do whatever it was that management told them they should be doing. I was given less than a week to find ways to make people that don’t want to work, work.

Basically, users fall into three simple categories:

Power Users – these users understand software, and require little, if any instruction – more than anything, you give these users a tool and stay out of their way.
Average Users – odds are, your mother or father falls into this category. They understand enough to get by, and with a little instruction they should have no trouble.
Idiots* – odds are, you work with one of these users. You lead them by they hand, and show them exactly what to do – just in time for the boss to walk by and praise them for doing a good job (and 10 minutes later you find them playing in traffic, somehow defying Darwin in the process).

For users of the last category, there’s just not much you can do.

I always do my due diligence while building software; doing all I can to make it simple to use, flexible, and forgiving of user error. I always use extensive data validation, carefully worded instructions and dialogs, and do my best to follow the various best-practice guides for UI and UX; yet for all this effort and design – I can’t write software that thinks for people or makes judgement calls based on business rules that only they know (probably because they make it up as they go).

No matter how helpful or intelligent an application is, or how idiot-proof you think you’ve made it, reality is that you simply can’t fix stupid – you can’t take an incompetent person that refuses to think for themselves and turn them to into a great, productive asset. After years in this industry (which has made me just a little cynical [in the way that Sol only seems little when compared to Betelgeuse]), I’ve come to understand something rather disturbing: idiots keep getting better.

Somewhere, right now, idiots are working to build even better idiots – and that’s a really scary thought.

We can make a user more efficient by automating tasks, providing better information, or helping to manage their workload – what we can’t do is make them smarter, make them think through their actions, or force them to do what their managers tell them. Yet we are, at least on occasion, asked to fix this problem.

Despite our best efforts as professionals and passionate developers; if a user won’t think – we just can’t fix it.

* – I define an idiot about the same way I do someone that’s lazy; they have no medical issues or legitimate handicap. They just don’t want to think or work (probably both). Those that are handicap or have learning or medical issues are a very different story and not the target of this article; I donated time and services to charities that served the disabled for a number of years, I highly recommend that all developers do it – it’s a very rewarding experience to see your work make somebody’s life better and it teaches you quite a bit about how people interact with technology.

bbPress & WordPress 3.1 (Registration Issues)

Adam Caudill — Sun, 06 Mar 2011 08:39:49 +0000

While building a new website for a small ISV, I was asked to integrate a forum – should be simple I thought, I’ll just use WordPress as a CMS and bbPress. Simple? Not quite.

To make things easier, I used Platform Pro by Pagelines (a great platform IMHO) and so used their “deep integration” instructions (while they are intended for Platform Pro users, the steps are virtually identical for all other setups). While the setup went quite smoothly, it wasn’t until it came time to test user registrations that the issue came up: when a new user tried to register they would receive an error stating that “email is required,” even if they had entered a valid email address. This effectively made it impossible for users to register, and thus made the forum useless.

After a bit of research, it looks like there is a conflict between newer versions of WordPress (3.0+) and bbPress 1.0.2 (I’m not sure if older versions have the same issue, but I would assume so). The fix for this is actually quite simple: install a plugin. The plugin itself is quite simple – containing only around a half-dozen executable lines of code, and works around the issue with a simple hack.

Unfortunately, the plugin description is rather vague and required a fair bit of research to identify as the fix, and the official documentation makes no reference to the issue or that this plugin can be used as a workaround.

Overall, a simple workaround but a major issue if you don’t notice that it’s broke (as is quite easy to do).

Rails 3 & Dreamhost PS

Adam Caudill — Fri, 28 Jan 2011 07:47:01 +0000

I recently had an idea for a small web application, and seeing as I’ve not spent as much time as I’ve wanted to using Rails – I opted to build it the latest version of Rails. A decision that caused far more grief than I expected.

If you are using Dreamhost’s PS offering (a managed VPS for those that don’t know), the seemingly simple task of getting a Rails 3 application up and running is actually quite complex. The root cause of this is that Dreamhost’s OS image is based on Debian etch, which was released in April 2009 and has since been replaced; which means etch has become fairly outdated.

Here’s the process I used, and so far it seems to be working quite well:

Domain Setup:

When adding your domain to the Dreamhost panel, you’ll want to enable Passenger.

Once your application is uploaded to the server, you’ll be greeted with a particularly unhelpful error message (something like “uninitialized constant Bundler“) from Passenger (or perhaps just a 500 error page).

Server Updates:

This is where the work starts, and gets somewhat ugly. As a warning, it’s quite possible that you could damage your configuration doing this; though thankfully you can restore your server to a working state within a few minutes from the Dreamhost panel should something go wrong. You’ll also need to have an “admin user” for this task, as much of what needs to be done has to be done as root.

First step: Get your PS up to date; even after performing a restore on my server, there were a number of updates that are available to be installed. So let’s start off by getting those out of the way.

sudo apt-get update
sudo apt-get upgrade
sudo apt-get -f install

Once you get past those three commands, the next step is to update SQLite to the latest version, as the version Dreamhost uses is quite old and won’t work with Rails 3.0 (well, to be accurate it won’t work with the latest version of sqlite3-ruby, which is the default database provider for Rails 3).

wget http://www.sqlite.org/sqlite-autoconf-3070400.tar.gz
tar zxvf sqlite-autoconf-3070400.tar.gz
cd sqlite-autoconf-3070400
sudo ./configure --bindir=/usr/bin --libdir=/usr/lib
sudo make
sudo make install

If you don’t update SQLite you’ll get an error like this:

sudo gem install sqlite3
Building native extensions.  This could take a while...
ERROR:  Error installing sqlite3:
	ERROR: Failed to build gem native extension.

/usr/bin/ruby1.8 extconf.rb
checking for sqlite3.h... yes
checking for sqlite3_libversion_number() in -lsqlite3... yes
checking for rb_proc_arity()... no
checking for sqlite3_initialize()... no
sqlite3-ruby only supports sqlite3 versions 3.6.16+, please upgrade!
*** extconf.rb failed ***

or if you install the updated version, but don’t force it to /usr/lib you’ll get an error like this:

sudo gem install sqlite3
Building native extensions.  This could take a while...
ERROR:  Error installing sqlite3:
	ERROR: Failed to build gem native extension.

/usr/bin/ruby1.8 extconf.rb
checking for sqlite3.h... yes
checking for sqlite3_libversion_number() in -lsqlite3... no
sqlite3 is missing. Try 'port install sqlite3 +universal' or 'yum
install sqlite3-devel'
*** extconf.rb failed ***

Once that is taken care of SQLite, the rest is easy.

sudo gem update

At this point if you visit your new Rails site, it should be working!

Notes:

I’ve not tested this extensively, and I’ve no idea if this breaks anything. All I can say for certain, if that all of my sites still work, but your mileage may vary.
I was a fairly early Dreamhost PS adopter, and part way through this process I reset my server to get it back to a clean state. After resetting, I noticed some differences with the behavior of apt-get (404s on update and upgrade are gone), so for other early adopters it may be necessary to perform a reset to get your servers configuration in-sync with the latest official setup.
I can’t say for a fact that this is completely necessary, though you’ll likely need to selectively update a few packages if you skip this step. Also, for me, gem was broken until I ran sudo apt-get -f install.
Special thanks to Matt for helping me get this working; troubleshooting the SQLite install was more than a little time consuming.

Google Chrome and H.264

Adam Caudill — Wed, 12 Jan 2011 07:00:42 +0000

In a somewhat (but not entirely) surprising announcement, Google is removing support for H.264 video from Chrome. This change to their implementation of the often controversial HTML5 tag is both a major step by Google and a furtherance of the already complicated world of video online.

… To that end, we are changing Chrome’s HTML5 support to make it consistent with the codecs already supported by the open Chromium project. Specifically, we are supporting the WebM (VP8) and Theora video codecs, and will consider adding support for other high-quality open codecs in the future. Though H.264 plays an important role in video, as our goal is to enable open innovation, support for the codec will be removed and our resources directed towards completely open codec technologies.

When Google released WebM (a royalty-free codec which Google acquired as part of On2), it was clear that the intention was to take on H.264 and with this move there seems little doubt that the gauntlet has been thrown down. Although, now that H.264 has such a strong base (it’s included in Flash, IE9, Safari, Mobile Safari, and Android), it really makes one wonder if Google has picked a fight that has long since been lost.

So why would Google do this?

Here’s my best guess: Money (specifically, patent licensing).

H.264 is heavily encumbered by numerous patents owned by companies like Microsoft, Apple, and Cisco, and controlled by MPEG LA, the consortium charged with turning these patents into profit (here’s the 70 page list of patents for those interested). While I’m sure many people recall that MPEG LA made a very public pledge that H.264 would be free forever, as is often the case, things aren’t quite that simple.

Peter Csathy wrote a fairly detailed post on the matter, pointing out some key details that many in the media skipped. Here’s the core of what wasn’t discussed after the MPEG LA announcement (but should have been):

But, you say, MPEG LA recently announced that it will no longer charge royalties for the use of H.264. Yes, it’s true – MPEG LA recently bowed to mounting pressure from, and press surrounding, WebM and announced something that kind of sounds that way. But, I caution you to read the not-too-fine print. H.264 is royalty-free only in one limited case – for Internet video that is delivered free to end users. Read again: for (1) Internet delivery that is (2) delivered free to end users. In the words of MPEG LA’s own press release, “Products and services other than [those] continue to be royalty-bearing.”

Mike Shaver, Mozilla’s VP of Engineering offer’s a somewhat similar take in “Free as in Smokescreen:”

What MPEG-LA announced is that their current moratorium on charging fees for the transmission of H.264 content, previously extended through 2015 for uses that don’t charge users, is now permanent. You still have to pay for a license for H.264 if you want to make things that create it, consume it, or your business model for distributing it is direct rather than indirect.

What they’ve made permanently free is distribution of content that people have already licensed to encode, and will need a license to decode. This is similar to Nikon announcing that they will not charge you if you put your pictures up on Flickr, or HP promising that they will never charge you additionally if you photocopy something that you printed on a LaserJet.

I’m just waiting for one of the licensors to reinterpret the license and claim that ads constitute a form of payment or some similar excuse to exclude them from the exception they granted. I’ve yet to get my hands on the latest licensing agreement to see exactly what it says about this, but I wouldn’t be surprised at all to see this card played at some point to wring extra revenue from these patents.

Given that Google owns the massive video sharing site YouTube, which uses H.264, plus whatever unknown projects relating to Google TV – it stands to reason that Google would certainly save some money by moving away from such an encumbered technology; not to mention avoid future risk should rules change. Though personally, I also have to wonder if it could be fears of a repeat of the GIF patent debacle.

Now where does this leave us?

Right now HTML5 is a mess, at best. There is a war for which codec becomes the de facto standard, and there is a lot of money at stake depending on who wins. At this point there is no single codec that works across all major browsers; to get full coverage the best option now looks to be a combination of H.264, WebM, and Flash. Doesn’t really sound like the progress that was promised with HTML5 does it?

It’s worth noting though that Google isn’t the first browser developer to reject H.264; both Firefox and Opera have decided against including it in their browsers as well. As painful as the fragmentation is now in regard to who supports what, this move by Google actually does little to change the landscape. Support has been fragmented from the beginning, and all this really does it push H.264 a step away from being the de facto standard; a title that it has been very close to seizing.

Had Firefox added support for H.264, I think the fight would be over and would have made today’s announcement almost suicidal for the project. Though with such a major player holding out against it, Google’s move becomes a minor tactical shift in the short-term (though the long-term impact could be significant).

I could go on for pages about what works are where we are now, but Mark Pilgrim (an infinitely better writer than I) sums it all up here: “Dive into HTML5: Video on the Web” – well worth reading if you want to really understand what’s going on.

So in summary – video needs to be encoded to multiple formats, which today’s announcement does little to alter due to the fragmentation that was already in place. In the long run, WebM may be better for the community due to its license, though many of the internet’s biggest players have a vested interest in H.264. So when you factor in politics and propaganda between competing companies, distrust, and possible patent claims that haven’t been addressed yet; this all leads me to an even simpler summary:

Yesterday, HTML5 was a mess; tomorrow it will still be a mess.