Shaw UBB Followup
While the proof is still going to be in what they do with these discussions, I was happily surprised at how the Shaw UBB session I went to played out. Now, it's important to remember the purpose of this sort of discussion for a company. I would be incredibly surprised to see any exact implementations come out of any of what was suggested, even though the folks running the discussion seemed very enthusiastic and encouraging over most ideas. One of the major challenges for any business is to figure out the right mix of choice vs convenience. And it was very clear from hearing what other customers had to say that ideas of fairness and what constituted acceptable use varied. Sometimes a lot. And I think that some of the ways in which they varied were quite interesting.
The one point that everyone on the customer side seemed to agree upon was that transparency was essential in the process. Shaw doesn't think that the figures from folks like Netflix and Michael Geist, regarding the last-mile Internet delivery costs being anywhere from 1-3 cents per GB, are quite right. I had in my head the 10 cent or less per GB cost of Amazon EC2 machines. As someone else pointed out, adding more capacity for Amazon in this case involves running a cable across a room, whereas adding more capacity for consumer internet potentially involves laying fibre over miles.
Shaw's own introduction was an interesting look into the challenges of providing residential Internet. They're looking at having to do around 500 "node splits" this year. What's that? Well, you have cable going into houses, but the signal starts to get distorted with cable after a short distance, so you have that cable going to nodes backed with fibre optic lines. Those nodes can only handle so much before another one needs to be put in. I'm not sure whether that's due to the fibre optic bandwidth itself or the processing capacity of the router. Regardless, at that point, if I remember correctly, they'll "split" the node, doubling the capacity and moving half of the residents to the new node. Those nodes go to more and more central stations along the way, which eventually link to the vast web of systems we all know and love. Now, putting in another node means obtaining permits, equipment, and not-exactly-plug-and-play installation. From the decision to "split" to the actual implementation it sounded like anywhere from 6-8 months until the thing would actually be operational. Certainly not trivial by any stretch. From their perspective, they're worried that next year it'll be 1000 splits, the year after 2000, etc. They saw a 60% increase from July 2010 to now. They feel this is unmaintainable.
I work with algorithms and software design. If anyone has a better understanding on the above, please enlighten me. I'm sure there are more than a few errors in what I picked up on.
So that's their side of the story, and they laid it out rather compellingly. I walked in with the impression of the big cable companies and telcos sitting on their thumbs, watching the money roll in and crying that the sky is falling. I left with the impression that they were sincerely trying to keep up with demand.
That said, there was a lot they didn't say and/or couldn't answer. Where did congestion tend to be a problem, for example? At the nodes or more centrally? Was it in the capacity of the fibre itself or the speed of the switches? It makes a bit difference whether you have to lay a bunch of new fibre to double capacity as opposed to letting Moore's law do it for you with better components at the nodes. Was it their efficiency in processing/routing traffic at the more central hubs? Problems there can be solved by smarter design and opening up to competition like the telcos have been forced to do. So while they laid out a compelling argument that it's not easy keeping up with network demand, there was still a lot of room for debate.
Back to transparency. At the end of the night, this was surprisingly the thing that they seemed to feel they would have the most trouble with. I had walked in thinking it would be the easiest. If you can give me numbers that I can independently verify, showing what it costs you to deliver me that GB, then I can decide for myself whether or not you're being fair. I don't have to just take your word for it. And I'm more than willing to pay for my usage by the GB if I feel I'm being treated fairly and you're doing everything you can as a business to be at your best. We know Canada has different challenges than countries where the population is more dense. Just show us the numbers. From their point of view, though, keeping those numbers to themselves plays a big part in business strategy. And in this, they seemed a lot like most more established businesses these days. Transparency seems natural to us, but it's alien and worrisome to an organization. Hopefully they'll find a way to satisfy our need to verify their claims with their need to keep some stuff to themselves, though, as that seemed to be the single idea that united almost everyone in the room.
Going on to the other suggestions...
I found that a lot of people shared my feelings that if you've got a certain cap and you're under one month (and they've demonstrated that they can easily measure that), you should be able to go over by that amount the next month and not be penalized. It's really hard to argue that one, and they didn't try. In fact, they seemed to like the idea themselves. From reading up on other sessions, it has been a popular one.
Raising the cap to something like 250GB was also suggested, of course. For most users, that was equivalent to saying "don't cap me", at least for now. That's of course where my cynicism rears its ugly head, suggesting that something like this would just be too easy for us and too generous of them, given their initial offer. But who knows?
Not monitoring (or providing discounted rates) non-peak hour usage was an interesting suggestion. This would work for those of us who have started using online backup solutions and/or occasionally have to download really large files (full games, for example). Upon hearing about our use of online backup, they asked what we would think of if they provided an in house solution to that. A service like Netflix's was also suggested. I can see the benefit from their perspective. This would allow them far more control to make sure the service met their quality standards and keep it from interfering with more general Internet traffic. I do have monopoly-type concerns with that, though. There's a reason I prefer Netflix to Shaw on Demand. It's quite simply a better service. And though they might be able to copy it, I don't think they would have come up with it unprovoked. I want them to be a pipe. A very good pipe. And I don't mind if they do offer some of their own versions of these services. But I still want to feel choice. I'm intrigued by the backup idea, but if, for example, it didn't offer client-side encryption and encrypted-only storage (so that not even an employee could see what's stored), I wouldn't use it for backup. There's just too much of a chance for a bad employee to compromise personal data unless you're the only one holding the key. I'm able to choose between Dropbox (great interface and tools, but no real encryption - besides creating your own encrypted image and storing that) and Jungle Disk (not a great interface, but client side encryption) and any number of other similar services out there. If the cost of transferring data is prohibitively expensive in choosing anything other than a Shaw solution, that's not enough choice.
Decoupling speed from usage was also brought up, which was something I hadn't considered. Discussion of it also led to, in my opinion, a much more nuanced way of handling network congestion. I feel it's because it gets to the root of the "capacity" problem. See, the problem isn't so much in the number of bits as the number of bits being transferred at one time. That's obvious, you say. But that isn't the problem that caps would be solving. Someone listening to Internet radio all day is probably not hurting network speed for others on the same node, even though when the month is over, they could well be considered a "heavy user". Meanwhile, the person who decides to download a bunch of huge files at a peak hour could be causing a disproportionate strain on the system. The latter may be under their cap but be the actual source of the network capacity issues.
So, is it more important that any time you download something, you can get it at 100Mbit/s, or would you rather be free to use 500GB throughout the month at a rate that's going to be somewhere between 5-25Mbit/s? Personally, I'd be happy with the latter. Gamers or DB admins who need to transfer large database dumps might prefer the former. The one-size-fits-all approach is still necessary for the vast majority of Internet users, as most won't want to think about it. But more customization would allow customers with highly specialized needs to get what they need without being considered a scourge on the system. The guy who's using 100Mbit/s at peak hours and downloading a terabyte per month may be out of luck until that becomes the norm, but most people would probably be able to find something that suits them in the meantime.
Shaw Consultation about UBB
Oh the drama of the "usage based billing" debate! Or is it really a debate when one side is a few huge companies with a virtual monopoly on high speed internet and the other side is pretty much everyone else?
The sparring match between the Prime Minister and the CRTC seems to be concerned more with the telco providers than cable providers, as the latter falls under slightly different rules. Also, the CRTC ruling doesn't change anything about what rules can be imposed on end users – it's concerned with the relationship between the telcos and independent providers which need to use the same infrastructure. So it was a nice surprise when Shaw took some initiative and scheduled some discussions about their recent decision to start imposing bandwidth caps.
I decided to take them up on their offer of attending a consultation session and thought I might get a little feedback from my millions of readers on what I should bring up. I'm walking in assuming that they'll see "no caps!" as a non-starter for debate, so with that in mind, this is what I plan to suggest:
- A more reasonable overage charge. If Amazon lets me run a server at ten cents or less per GB of bandwidth, why does my ISP think charging me $1-2 per GB is fair?
- Allow banking of GB under the limit. Most months, I'm using less than half my cap. This month I changed online backup providers and went way over. But throughout the year, it would even out. I think a lot of people are in the same position. Even fairer would be to get money back for the GB you don't use, but somehow I see them less likely to go for that. Businesses like the subscription model for a reason: it makes them a steady income per customer.
- A higher cap. I seem to remember hearing that high speed internet in the U.S. tends to have around a 250GB cap (if anyone has a reference to this, I'd appreciate it!). This seems a lot more reasonable given the rise of services like Netflix, that cause people to legally use a lot more bandwidth. In fact, Shaw's initial decision to start imposing caps shortly after Netflix was introduced in Canada was rather suspicious, as the service obviously competes with Shaw's TV offerings.
- Netflix and Michael Geist have both talked recently about the real cost for last-mile internet delivery (which would presumably be the part where congestion was really a problem). The figure ranges somewhere between one and three cents per GB. If ISPs are claiming that UBB at $1-2 / GB over the cap is fair, they should provide some publicly available numbers to back that up. Otherwise, they shouldn't be surprised when customers think they're just trying to milk them for more money.
Anything I should add? Any corrections to make to that information, or ways that the argument could be strengthened and/or made more persuasive (recognizing the fact that they're in it to make money and are going to be balancing how much the anger of customers is going to cost vs. the amount any concessions are going to cost)?
No, I'm not expecting much from this, but as with voting, I feel that an attempt at positive participation secures a certain amount of the right to complain afterwards. Please get any comments in before 6PM on Monday, March 7th, as that's when I'll be attending. I'll be sure to report back on how things go.
Brevity, my elusive friend…
I have fallen prey to one of the many clichés of blogging, developer friends. In the beginning, there seem to be far too many spectacular ideas and so little time, but before you know it, there's just this nagging thought in the back of your mind: "Oh yeah... I haven't done a post in a while, have I?"
Now, there are always several reasons any good procrastinator can give you for putting things like this off, but probably the biggest for me is the apparent inability to be brief.
Just today I told myself I was going to fire off a concise email about a design issue I wanted to discuss with my co-workers. Straight. To the point. Just the facts, m'am. And, after about a screen length of text, I figured I'd pretty much covered it.
Was it helpful for me? Undoubtedly. Writing helps me to work through ideas. I notice problems and solutions more easily than if I was just sitting there thinking. But did my co-workers need to go through the same thought process? Probably not.
When it comes to writing a blog post, something I don't really have to do, knowing that I'm probably going to go on forever about something is a good recipe for stalling.
On the art side, I've been exploring setting arbitrary limits in order to be able to still make stuff in this world of infinite possibility. I think I'll apply that to this blog as well. So here goes: this and the next 10 posts will all be under 300 words, no matter what. There, I've said it. Give me brevity, or give me backspace! (272 words)
Eclectic Ruby, MIDI, and Dr. Von Funk
One of the things I like about the Ruby community is that it seems to attract and even encourage eclecticism. I suppose it was only a matter of time, then, before a book like Practical Ruby Projects: Ideas for the Eclectic Programmer was written. I'm interested in particular in the chapter on using Ruby to drive MIDI. Someone's even hooked it up to Reason. This is, of course, awesome.
So, my grand plan is to turn my iPad into a step sequencer for Logic Pro. To do that, I'm going to need to figure out how to send MIDI signals to it, but also how to receive MTC from it. Even better would be to figure out how to send a note with particular timing information (i.e. "Play this note here, and this other note there, etc." instead of just waiting for a particular time and saying, "Play this note now!"), but I'm not sure that's possible at the moment. It's kind of daunting when I haven't done any audio programming before, but Ruby makes it a little less daunting, because it allows you to strip away a lot of the hassle of programming and get to the point quickly. One of the hardest things with any project is proving to yourself it might be possible. Possible enough, at least, to start hacking away at it.
Also, I started a new job. I have a view with a window, and I get to make software in Ruby on Rails that helps people make cute cartoon characters to help welcome and guide other people through cyberspace, all the while avoiding the perils of the uncanny valley. That's a lot of helping! Everyone here has a window seat, which is great because I've never really had a good view before (well, once I did, but the whole time I was there, people would walk by and say, "How the hell did he get that desk?" and I never really got to enjoy the view because I was too busy trying to figure out what it was that I did – apparently whatever it was, I did quite well at it, which was somehow even more unsettling). So having a view that everyone else has that can give your eyes a break every so often from the computer screen is great. Working with other people in the same physical space is also great. I didn't realize how much I'd missed that until now.
Here's something (I think) I'm learning about creativity. I'd spent several years looking for more and more freedom. Less structure, more blurry lines, less certainty. That's good to a point. I think it helps you to be creative if you're comfortable with those things. But creativity can also use a good dose of structure and space every now and then. Walking to work. Coming and going at regular hours. Glancing up at the awesome view when you've hit a roadblock. Being able to leave work at work. I'm also working with a great team that I think I'll learn a lot from and contribute a lot to. My girlfriend keeps asking me, "Are they the ones behind The Paperclip?" I don't think so, and that's also nice, because I think enabling The Paperclip and its plans of world domination would be much more taxing than working with cute cartoon characters like Dr. Victor Von Funkalicious.
What my dad taught me about my career in software development…
My dad knows a little HTML. That's a lot more than most people his age. But whenever I get too involved in relating something about what it is that I do for a living, he just smiles and patiently waits for me to finish. And I realize that while I might be entertained by whatever problem or design it is that I'm talking about, he's entertained by how involved I get in it – how much it may anger, irritate, concern, or inspire me. And while university and my career may have taught me things that he doesn't (and likely doesn't care to) understand, I'm just beginning to understand what he's been telling me for years. Like a lot of kids in the west, I've probably spent more time differentiating myself from my family than finding similarities with them. The need to constantly prove your individuality in our society might be better classified as a sort of neurosis instead of something we should aspire to. But the more I think about it, the more I realize that the traits that have helped me most in my career, all the way from learning my trade to the day to day business of it, were – you guessed it – taught to me, very patiently, by my dad.
So without further ado, here are ten of those things:
- You won't learn if you don't get your hands dirty and do the work.
- Just because you don't understand it, doesn't mean it's not important. That said, you don't have to understand everything RIGHT NOW.
- Admitting your mistakes promptly, endeavoring to understand them as fully as possible, and taking steps, however small, to learn from them will serve you far better than making no mistakes at all. People will also respect you more.
- Doing what makes you happy is the most practical and down to earth way you can live. That doesn't mean you won't ever have to make what you've been convinced to think of as a sacrifice in order to do what makes you happy. (Both my parents used to tell me they'd be just fine if I chose to dig ditches for a living, as long as I was happy. And I wouldn't knock ditch diggers. The movers and shakers on Wall Street may get to live a much more comfortable lifestyle, but at the end of the day, I'd rather lay claim to a ditch that served a purpose than a fat salary that helped drive the world economy into recession).
- You can only control what you put out into the world, not how the world chooses to react to it; however,
- You can and should consider the consequences, even if they don't change your mind.
- Frustrated? Stressed? Life treating you unfairly? Try and make it easier for the next guy who finds himself in your shoes.
- It's easy to let ego get in the way of happiness. Try not to. (Still learning, Dad).
- Speak your mind and do what needs to be done. Sincerity and a good work ethic will serve you better than brains and luck in the long run. The right people will understand, even when they don't.
- Your career might be what drives you, but it's your friends and family who give you the fuel to do it and who help you fix the things that occasionally break in your life. Ignore that to your own detriment. (Still a novice here, too. Fortunately, my friends and family are very forgiving).
- Question authority (politely, if authority has a badge and gun). Rules aren't ends in and of themselves – they're means. It's okay to break some – even your own – when they stop making sense.
As I'm sure you've decided by now, this post really has nothing to do with software development. Or does it?
iPad: First Impressions
Despite not being completely blown away with it (the "it's indistinguishable from magic" propaganda from Apple was a bit much, even though I think the company is probably still the most innovative out there when it comes to making tech products that are actually a joy to use), I decided to line up for the Canadian debut of the iPad. From a pure software developer perspective, it's hard to imagine the future not being filled with more and more touchscreen devices like this, and I've had a few ideas rolling around about apps that might be cool when used with the larger screen (over the iPhone) that the iPad gives. Maybe I'll even be able to beat the expected dilution of the market as the app store gets flooded with iPad apps to do just about anything. (Need to walk your dog, but too lazy to do it yourself? There's an app for that!)
And, admittedly, although I don't share the love of Star Trek that has become part of the caricature of your average software developer, I do find it hard to resist getting to know a new gadget.
So now that I've had a few days to play with it, here's what I do think is amazing about the iPad. Here's the spoiler: it has nothing to do with the hardware.
That's not to diss the people who actually make those electrons flow down the proper channels in such a way that I'm able to hold a tiny screen in my hand for ten hours per charge that makes the room sized computers of fifty years ago seem like a joke. It's just not my domain. I don't see the value of objects in and of themselves. But if we can do something really, really cool with them, well that's when it starts to get fun.
And I think this might be why Apple's iPad has been met with a more dimmed enthusiasm than previous products. It probably doesn't have any more or any less innovation involved than any of their previous products had at the time. It just doesn't seem like as much of a leap. Why is that? I think that a big part of it this time around is that none of the software that Apple has bundled with the iPad is that much different. Hell, they're even using the same operating system that they use for the iPhone with it. I can have an extra column when I use email? A fuller looking calendar? I can read an eBook on it? Well, woohoo.
Then I decided to pull down the first iPad edition of Wired Magazine and I saw the full potential of what devices like the iPad might bring to at least one very troubled industry.
It's mainly about the advertisements.
I found myself almost more interested in them than the various articles. If this isn't the moment for the advertising industry to lead a full charge away from old media forms and onto the Internet, I don't know what is. While everyone uses television commercial breaks to go to the bathroom or refill their drink, the ads in Wired magazine actually made you want to poke around and learn more. They were able to prove you with a simple and elegant, non-intrusive introduction. No blinking neon signs here. Just flip the page to be onto the next article. But maybe something caught your eye? Well, there's a play button you can press to see a video version of the advertiser's message. And if you touch some of the headings, they'll pop up more info for you. Just if you're curious. You don't have to. It's up to you. And hey, if you're really, really interested, here's a link that will take you to their website to actually buy the thing.
Wow.
It's like being able to have an entire website of information about your product embedded into a single page of a magazine. The purpose of advertising is to interrupt your routine. And that's not always a bad thing. Imagine if you had to actually go out and search for that brand new technology that will make your life so much simpler. How would you even know what to search for? While convincing us of needs that we didn't even know we had can be predatory, and in the current world of advertising it often is, there is nothing inherently evil about wanting to inform people about something they don't know about. What ads like these allow is for the advertiser to catch your attention with the typical "this will change your life" sort of appeal. But it also allows them to be more honest and give you more information right away so that you can decide for yourself how much your life will change and whether it will actually be for the better. I'm not so silly as to be utopian about this and say it's the end of deception in advertising. But I do believe that it provides an avenue for companies to get their message out in a way that doesn't litter our day-to-day lives so much. And that's good for everyone involved.
I imagine there'll be more than a bit of money out there in making these sorts of content rich ads easier to produce and embed. And I don't think there's much argument anymore that the Internet friendly ads like those in the e-edition of Wired are more valuable for advertisers and consumers alike than their old-media predecessors. It must be a heady time for the ad networks to consider the exciting possibilities ahead of them if they decide to go for quality over quantity.
So what else am I interested in seeing on the iPad? Textbooks. Imagine being able to open your calculus textbook, look at a graph it uses to explain a concept, and actually change values to see what changes on the graph? Or to be able to step through a virtual chemistry experiment? How about seeing the effect of your incorrect physics solution on the orbit of a space shuttle around a planet? I think that this level of interactivity could have an amazing affect in the area of education. And who out there doesn't believe that this is an area that we need to pay more attention to?
It's never the hardware that changes the world. It's how we decide to use it. And I can't think of any other device at the moment where this is more apparent than in the iPad and it's soon-to-be brothers and sisters. They could easily be a throw away fad or the future of computing, depending on whether or not they're able to capture the collective imagination of the software development world...
Caveat:
As this article points out, the Wired app may be, technologically speaking, a horrible mess. I agree that simply slicing up images to build your interface when you could be using a lot of HTML 5 to get the same effect is incredibly kludgy, and the amount of space that requires for an app is unacceptable in the long term.
However, I disagree profoundly with the idea that it was a mistake to move away from a more browser-like experience for the e-version of a magazine. I think us techies are sometimes too dismissive of the importance of how something looks. The success of Apple in recent years should have proven that to some degree, but maybe it's one of those never ending arguments. I've read other articles that suggest it's a step back that the Wired app doesn't try to more closely resemble a regular web browsing experience.
But I'd suggest that we shouldn't be so quick to say that the way information is organized right now on the web is at all optimal for every situation. In fact, an article in the current edition of Wired (which, granted, I couldn't have linked to in the app version) points out what I think anyone who still enjoys sitting down to read a real book or even magazine article will intuitively know.
The fact that we tend to follow links, etc. while reading content online, often only skimming content for something in particular that we're interested in, can result in a much more superficial form of learning. To be sure, we're able to get many more viewpoints around any particular issue, and thus get a more well-rounded perspective on it (and I think this is a very positive result of the web), but we're also less willing to sit down and give a lot of time to considering a single viewpoint before going elsewhere. There is a value that you get from sitting down with a single work (be it a novel, essay, or whatever) from a single mind, and giving it your full attention that you can't get in any other way. It may not seem as useful in the fast paced world of the web, but I think we ignore that at our own peril.
So I actually like the fact that the Wired App presents a unique and visually engaging experience that makes me want to actually sit down and read an article all the way through. It doesn't leave me feeling fidgety, wondering what else is out there on the Internet that I'm missing out on because I'm focusing all my attention on this one thing. And I hope that as we bring the experience of reading things like novels more into the electronic realm that we consider this. Yes, it's great to be able to cross-link and do all manner of Web 2.0 stuff with a novel. But I don't want that stuff to be in my face. I want it to feel like I'm reading a book most of the time. I think that's valuable. I like the idea of giving a designer control over everything, even the font face, so that he or she can present their full vision to me for a given publication.
There's a lot of thought that goes into laying out a magazine or even a novel for printing. Just go to any bookstore and pick out a pulp book to compare to one from an author du jour. Don't look at the quality of the writing. Look at the quality of the presentation. Taking away anything about the actual content (i.e. remove the actual author and subject matter from the picture). Which one's easier to read? Probably the one with the bigger budget. And that doesn't mean you need a big budget. It just means that considering those minute details involved in choosing typefaces, line spacing, layout, page-breaks, etc. (which tends to happen for books with a bigger budget) makes a difference. It's nice to be able to change fonts, etc. on something that's poorly laid out. But I'd rather just get a good quality layout. If I'm still paying $10 or more for an e-version of a book, I actually expect it. A plain text document might give me all the power in the world, unless my interest is actually reading the thing.
So, yeah. Can Wired on the iPad use some improvement? Certainly. But I still think it's a step in the right direction, especially for an avid reader, not to be confused with an avid web-surfer, although you often find people who fall into both groups.
On copyright
I saw this on BoingBoing earlier this week and thought it was especially appropriate as my own country moves towards adopting a stricter set of copyright laws. In summary, the presenter cites the fashion industry as an example of a world that not only survives but thrives under a complete lack of copyright protection. The thing that I think makes this argument especially compelling to me is that the arguments for protecting fashion under some form of copyright are the same as those being used by most media-centric industries today – and yet the fashion industry is still doing quite well without these protections. I like the comparisons drawn between music sampling and fashion sampling. When sampling music, no matter how small the sample, you're legally required to secure the rights. In fashion, you can outright copy someone, and as long as you don't use their name on the product, you're not breaking the law. The presenter argues that far from stifling creativity, the allowance of this sort of sampling in fashion drives innovation.
As a software developer who has used and worked on both open source and proprietary software, and as a musician and writer who has spent many years and will likely spend many more honing his craft (who has also sold music and played for money, albeit chump change at the local level), I think I'm pretty qualified to say something about copyright. I believe that creativity is valuable and that it needs to be compensated. It bugs me when someone says that I should do it for free. Or that I must. Anyone who's passionate about creating things will do it regardless of whether or not he's paid to do it. If we abolished all forms of protections and in fact made it illegal to make money off of works of art, we'd still have works of art. It's not a rational economic choice that one makes to be an artist. But when you allow someone who truly loves their craft the ability to also pay their bills and maintain a reasonable level of comfort with it, everyone wins. You get better quality art simply because they're able to spend a lot more time practicing their craft.
So, I don't come from the Napster-flag waving crowd. I think creative people, especially artists, are undervalued and always have been. And I hope everyone keeps in mind that even some of the most famous and wealthy artists were once independents who went home after a full day of work and, instead of turning on the TV, spent another six to eight hours sweating over their craft. If they're not deserving of at least your respect, I don't know who or what is. And if you think that what they produce is of value, you should find a way to support them, no matter how small. Yes, some people lucked out. Some people win the lottery too. That doesn't mean that there aren't a whole bunch of other people who actually deserve the credit they get.
That said, how does extending a copyright to a song seventy years after my death help me to create? Why should my heirs be able to live off of one or two very successful works, if I were to produce such things, without contributing anything of their own to society? Why should you have to jump through so many legal hoops to pay homage to another artist by sampling his work? And if you're a big fan of a band or movie, should you really have to risk legal action in order to help promote them by passing their stuff around to your friends?
That last action is, of course, classically referred to as piracy. And, to be fair, sharing music out of a hard drive and using it as a sort of currency to get other music you want (unless you're okay with the much frowned upon practice of "leeching," as it's referred to in the file sharing community), isn't quite the same thing as saying to a friend, "Hey, you've just got to hear this band! They're amazing! I'll send you a track!"
But who's fault is that? Any market that is made illegal (such as the file sharing market) is naturally distorted because of that illegality. You can't say that it would have exactly the same characteristics as a similar legal entity that had a few reasonable controls placed on it. For example, if people weren't worried about being sued for sharing files, they'd be more open about it, and you could use that data to market to them and/or implement fee structures that everyone thinks are fair.
As an indie musician, even doing short runs of 500 CDs, it costs about $2 per copy. I know this because I've done it. For a few thousand dollars, you can have the equivalent of a million dollar 1980s recording studio in your home. I know this because I have such a studio, and I've watched over the years as the sound that only the pros could afford fell further and further into my price range.
Fans trading music around can end up doing your marketing for free. Most indie artists I know would love for people to start "stealing" their music. Our enemy is obscurity, not piracy. Get the obscurity thing out of the way and most indie artists can do just fine on their own.
So where do those extra $13 per CD really go? That markup might be necessary, but you have to be able to convince your customers of that. Notice that I said "convince them of that," not "compel them to pay that." When people see indie musicians producing often more inventive music at the same or lower price (when these musicians are not able to take advantage of the economics of scale), you need to find a way to justify your own price.
In my mind, the biggest contributing factor to piracy and its more harmful aspects, is that we've barely even tried to recognize the new digital reality. The music industry is just starting to. It took them over a decade. How long for the movie industry or the book industry? Or the manufacturing industry? I can't wait until machine fabrication becomes so accessible that people start trading design specs so they can print off a brand new pair of Nikes. We might as well start thinking about how we're going to compensate people in ways that encourage the flow of ideas instead of stifling them because the world is only moving further in that direction. There's no going back. Let's just accept that, embrace the future, and try to save what's truly of value in the present.
Copyright law, at least as it was applied in the nineteenth and twentieth centuries, just isn't compatible with what's happening. We need to accept this and move on, instead of tying ourselves in administrative knots. And somehow, it seems, we need to still convince our leaders (in industry and in government) of this. Maybe we need to buy them all iPods or something.
Because instead of communicating and adapting, they've gotten the lawyers involved – and nothing drives up the cost of anything creative like getting the lawyers involved.
I have the maturity of a twelve year old… and so do you.
WARNING! MUCH TECH-SPEAK AHEAD.
So, there's a certain type of test that's been failing in the M7 code, and I've been working on it all day long. It has to do with the Erector 0.5.1 gem from Pivotal Labs. Now, they've since moved on in versions (all the way up to 0.7.3), but for various reasons, I'd like to avoid upgrading the gem right now. (I'm pretty sure I remember hearing there was an issue with upgrading Erector sometime in Week #1, and regardless, it was an upgrade that got us into this test fixing party in the first place – I don't want to add any complexity by upgrading more stuff that will certainly break existing code before I've dealt with these changes). Plus, I don't even know for sure if the upgrade would fix the problem we're having with these particular tests.
I think this is a good time to pause and let you all snicker a bit (yes, dad, you too), catch your breath, compose yourselves, and... can we move on now? Are you sure? (Thank you, WordPress, for giving me the power to moderate comments!)
Well then, moving on.
So, Erector hooks into Rails' ActionController rendering via the following code:
ActionController::Base.class_eval do
def render_widget(widget_class, assigns=nil)
@__widget_class = widget_class
if assigns
@__widget_assigns = assigns
else
@__widget_assigns = {}
variables = instance_variable_names
variables -= protected_instance_variables
variables.each do |name|
@__widget_assigns[name.sub('@', "")] = instance_variable_get(name)
end
end
response.template.send(:_evaluate_assigns_and_ivars)
render :inline => ""
end
def render_with_erector_widget(*options, &block)
if options.first.is_a?(Hash) && widget = options.first.delete(:widget)
render_widget widget, @assigns, &block
else
render_without_erector_widget *options, &block
end
end
alias_method_chain :render, :erector_widget
end
This is all well and good, except that we also test views individually via RSpec, which means we have to fudge variables (on any Erector widgets we test that take variables) that would normally get set through the controller. When you're just rendering a template, Rails gives you a mechanism to set local variables at this point, via:
render 'some/template', :locals => { :some_local_var => 'some value' }
You'll notice up above that, instead of passing the options along when rendering the widget (in the case where :widget is given as the first option), the substituted rendering functionality passes the value of @assigns. This works when you're using the widget through a controller, but breaks down when you try to render the widget on its own.
So, when we call the following line in one of our tests:
render :widget => Views::ScriptVersions::Show, :script_version => @script
The variable script_version that's local to the widget does not get set to the value of @script, like we'd want it to. And trying various combinations of :locals => {} didn't work either (as it would on a regular template render. I was consoled by the fact that at least one other person seemed to be dealing with a similar problem in this area of the code. However, I figured I was unlikely to get a fix for it (and for that version of the gem), as the later versions seemed to address the need to set variables without using the controller (though I still wasn't sure how easy this new way of doing things would be to put into an RSpec test). Not an option unless I wanted to disrupt a bunch of code with the newer syntax, etc.
So, after a bit of digging through the rspec gem code, I realized that the Spec::Rails::Example::ViewExampleGroup class, which the various rspec view tests inherit from, did contain a reference to a controller in order to do its magic without us needing to explicitly set one up. And that meant I could just set the @assigns local variable on that via some simple Ruby reflection magic (thank you, Ruby!) before rendering the widget, and the widget would pick up the variable as if it had been set in the controller:
controller.instance_variable_set(:@assigns, { :script_version => @script })
render :widget => Views::ScriptVersions::Show
And voila! All tests pass!
Ruby is really a great language to work in. It lets you do so many things that would take you much longer (and look much uglier) in another language. The price for this is that when something goes wrong, you have to decipher all the nifty meta-hacking, which is usually a great deal more time consuming than in other languages. Trade offs. You can never quite get rid of them. I sometimes wonder if the time one saves by coding in Ruby is completely negated by the amount of time he or she spends on treasure hunts like this. That said, the beauty of open source is that I was actually able to step through this code, line by line in the debugger (my new favourite thing to do!), to see what was happening, which is something you can't always do. I've run into similarly arcane problems while programming in Cocoa without that luxury.
Sometimes you just have to bang your head against that wall…
image from ilco on sxc.hu
Thursday night, there were 136 failing tests left on the M7 test suite. As of this morning, that number is 90. 46 fixed tests in 3 days. Not bad! Of course, 1.5 of those days consisted of trying to fix a set of tests that I never quite figured out (but fortunately, I have lots of notes and ideas!).
The career of a software developer is one of continually not knowing the answers. That's what makes the job fun. Every day, you're learning something new, solving some new problem (unless you're in one of those other software jobs, you poor soul!). But when you're racking your brains (I was corrected recently on this term – it's not wracking, it's racking) out on why Factory Girl (yes, this is actually a Ruby gem) won't work the same under Rails 2.3.2 using particular "associations" and there's only one way-too-simple example to go off of (nothing is said about creating a factory with more than one child association, which is unfortunately what we had) and not very much documentation, well... it can be a little frustrating.
But sometimes you just have to beat your head against that wall until you get it. It's a concept that most skilled creative types have to learn at some point. I remember learning guitar and it just seemed so hard to stretch my fingers far enough to make a power chord. Then one day after weeks and weeks of trying to do it, it just clicked. The process of writing songs is like that for me as well. I seem to make about ten or twenty failed attempts at one for every good one I write, and usually that one's written in under a day. But those ten or twenty failed attempts were as much responsible for that one good one as anything else. They attempts had to be made. I had to go through that frustration to finally relax and write the one that worked.
I've actually started to look for the odd brick wall. The thing about them is that they act as natural barriers to everyone else. When you look at the amount of stuff out there on the Internet, it's easy to get discouraged. Everyone has a voice now. Tools are being created every day to reduce the barrier of entry (in terms of skill) to a lot of areas of creativity. I actually think this is a good thing, as it allows people to exercise other areas of creativity. But if you want the creative life to also pay the bills, you need to continually stand out. Brick walls help you do that. Because they can be pretty daunting, and they won't seem worth it to most people. And, actually, they aren't really worth it, unless there's at least some passion backing your attack. You've got to love something about the challenge. You've got to be a bit obsessed.
And sometimes you've got to just keep going when you've got a bit of momentum. This was basically a work weekend, but I don't mind doing that the odd time. Of course, I also expect people to understand when it goes the other way a bit. Creative work does require discipline, but it also requires a little more flexibility than is built into the nine to five system. You've got to give time for brick walls, and also take time to rest a bit once you're on the other side. Still learning to properly do the latter.
Food for thought: I've been musing over this slightly related post over the last couple of weeks. I think there's a lot of truth to it. I hate the uncomfortable moments, but good things often seem to come from them.
The Art of the Confessional
I think you could make a very compelling argument that programming is simply the practice of making trade offs. Minute after minute, day after day, week after week, we're presented with these tiny little choices. The simplest choice in the life of a programmer is, of course, "on" or "off", one or zero. But few of us even need to get that close to the mind of a machine anymore. Languages like Ruby are often a joy to use because they provide such abstraction from that on or off world. By building abstraction on top of abstraction, they've given us a multitude of ways to express what it is we want those pesky machines to do.
But try as we might, we never quite get to that point where we can turn off our brains and let the machines do all the thinking – so that we simply tell them to make us breakfast (or some other absurd techno-utopianism) and have them serve up the perfect meal: exactly what we wanted, how we wanted it, efficiently delivered, and all that jazz. I mean, how would we even begin to do such a thing? First we'd have to figure out how a machine could figure out what you liked. Perhaps it could observe you and see that every morning you make yourself some toast, pour a glass of orange juice, and have a bowl of cereal (if this breakfast seems rather uninventive, it's because I hardly ever eat breakfast, which I know is very, very bad for me – but then we all do things that are bad for us, don't we?). And by observing the frequency and types of food, it could quite easily infer that a "good" breakfast to make would be one including toast, a glass of orange juice, and a bowl of cereal. But what if you were only doing that because you were a complete buffoon in the kitchen (like me)? What if you really wanted it to create exotic breakfasts from around the world for you? A new one every day! Well then, maybe we should introduce an element of randomness into this machine. Perhaps, every so often, it rolls the dice and picks a meal you haven't had before, no matter what it's observed. As a further refinement, maybe we'll let you vote after each meal whether you liked it or not, and the machine can keep a record of these and infer better random choices for future breakfasts from the data it collects... well, this is going to totally piss off the guy who just wants the machine to make him some damned toast and orange juice and cereal! It's the 21st century! Come on! This isn't too much to ask, is it? (Don't lie... you all know someone like this.)
Hopefully this gives any of you non-programmers a little more insight into how us programmers think (and to the programmers, perhaps it can give some good arguing material when your friends bug you about thinking too much about things – of course you think too much about things... thinking too much about things is your job!). As I've (hopefully) demonstrated above, we can even turn a simple breakfast into a war of algorithms. None of them is "the right way" to go. But each of them might be the best choice depending on your target user. Want to cater to grumpy old men who like their routines? In that case, the first, less complex, algorithm above is the way to go. But for the adventurous connoisseur (aka "the power user"), complexity is ideal because it means they have more switches and buttons to play with in order to tweak it into what they consider the perfect breakfast making machine.
But that's all still terribly high level for us programmers. Not even close to the necessary ones and zeros that are the only things machines will really understand (and even that, of course, is being very loose with the definition of understanding). The choices that keep us programmers awake at night are written in code.
One such typical code-based dilemma is the choice between hardcoding and making generic components. Especially with complex business logic, where it's sometimes arguable as to what constitutes the sinful act of "hardcoding" and what doesn't. After all, the ideal generic component is the computer itself. If everyone could program, we'd only really need one application, and that application would allow us to do absolutely everything given the right commands. So, at least sometimes, the more generic something is, the less user friendly it will be. And yet every programmer I know has at least in one point in their careers, striven towards this ideal generic code that will do anything and everything. It drives managers nuts. It drives us nuts (though it's an intoxicating sort of madness). And eventually, if we stay in the field, we learn to draw often arbitrary lines as to what's going to far and what's just being a lazy hack. Hard core religious folk ain't got nothin' on how heated these ideology debates can be!
Another common decision is between decoupling and duplication of effort, which is actually the thing that came up today at work that prompted this post. Here it gets to be even more fun and more difficult for your friendly neighbourhood programmer, as decoupling and the avoidance of duplication are practically stone tablet material when you're learning your trade. But it's pretty difficult to get both in ideal quantities. More often than not, you have to choose.
And at this point, you're wondering if I'll ever explain the title of this particular post. Well, here it is...
The point I'm trying to make above is that, although programming is often seen as a science where it's possible (if we're just smart enough, damn it!) to make a computer do exactly what it needs to for a particular industry or user, the real world calls on us to be artists (which is not so bad for those of us who like being artists) as well, who sometimes need to make choices they might not even be able to explain, based on time and circumstance. And yes, even very good programmers can feel very stupid trying to explain why they went one direction as opposed to another when suddenly all hell breaks loose and that other direction starts to look like the road to paradise. "Well, yes, with hindsight, that is certainly the way we should have gone! But you see, at the time, well, there were certain objectives and, well no, I can't quite remember what they were, but there was a reason we went this way in the first place, I swear!"
That's where "the confessional" comes in. If I've stolen the term from someone else, please accept my sincere apologies, but among the programmers that I know, I do believe I've at least led the charge in advocating it.
There have been several times in my career that I've faced some agonizing decision where I either wasn't sure which way was the best way to go, but had to pick one, or I simply saw the possibility of negative consequences down the road but knew there was not enough time to cover every possibility. And there were also times where I outright hacked something together because it had taken too long, I had too little understanding to make a proper solution (and no one else to ask), or the coffee wasn't strong enough to de-fuzz my brain that day. There's always a tendency to either be paralyzed by indecision or to make the choice silently and hope no one discovers your secret code-crime.
Avoid this. Write a "confessional". Every decent programming language gives you this beautiful structure called a "comment", which allows you to babble on to your heart's content about how frustrating a problem is or how you know this is an awful thing to do, but you really had no choice! Use it. Get it all out. You don't even have to make the machine understand it. You won't make the program less efficient (compilers usually strip these out, and in non compiled languages, it's easy to write something to do the same thing). And even the odd page-long confessional won't have your fellow programmers snickering behind your back about what a hack you are. Well, maybe. But you shouldn't worry about it.
No matter how long they've been (and I've had some long ones), I haven't once had a fellow programmer bring up one of my "confessionals" as criticism. On the contrary, people have actually said thanks at times, seemingly out of nowhere, long after I've forgotten writing it, because they were chasing down something to do with the problem or considering making some change that I had considered and decided against (but talked about in the "confessional") and by writing down my horrible coding transgressions and why I made them, I saved them from doing bad things they hadn't considered. Other times, I've run into my own "confessional" in trying to explain to a manager why something's the way it is and not quite being able to make a good enough argument. I suddenly hit the "confessional" and say, "Oh yeah, now I remember! It was because..." I really don't think managers care so much whether you make the odd bad decision. I think they're more interested in whether you care about what you're doing. And combining a "confessional" with something you see as a possible code-crime is a good way of showing that you care about what you're doing, even when you can't do it perfectly, for whatever reason.
Caveats. First, if you tend to write more "confessional" than code, you should perhaps consider another career. Just like swearing, a "confessional" is more effective when used sparingly. (I've always gotten a kick out of adults who desperately try to get kids not to swear, usually by admonishing them, with varying degrees of harshness, over it. I think a much better way of attacking this one is to convince them that a word like fuck is way more shocking when it comes out of nowhere... especially after 1694 much more innocent words!). The second thing is that, on longer confessionals, it's considered courteous to stick the odd joke (geeky in-jokes even better!) in there as a sort of "thank you" for someone else taking the time to slog through it all.
With those caveats in place, programmer friends, go forth and write wonderful code! Repent less, confess more, and be merry!