While the proof is still going to be in what they do with these discussions, I was happily surprised at how the Shaw UBB session I went to played out. Now, it's important to remember the purpose of this sort of discussion for a company. I would be incredibly surprised to see any exact implementations come out of any of what was suggested, even though the folks running the discussion seemed very enthusiastic and encouraging over most ideas. One of the major challenges for any business is to figure out the right mix of choice vs convenience. And it was very clear from hearing what other customers had to say that ideas of fairness and what constituted acceptable use varied. Sometimes a lot. And I think that some of the ways in which they varied were quite interesting.
The one point that everyone on the customer side seemed to agree upon was that transparency was essential in the process. Shaw doesn't think that the figures from folks like Netflix and Michael Geist, regarding the last-mile Internet delivery costs being anywhere from 1-3 cents per GB, are quite right. I had in my head the 10 cent or less per GB cost of Amazon EC2 machines. As someone else pointed out, adding more capacity for Amazon in this case involves running a cable across a room, whereas adding more capacity for consumer internet potentially involves laying fibre over miles.
Shaw's own introduction was an interesting look into the challenges of providing residential Internet. They're looking at having to do around 500 "node splits" this year. What's that? Well, you have cable going into houses, but the signal starts to get distorted with cable after a short distance, so you have that cable going to nodes backed with fibre optic lines. Those nodes can only handle so much before another one needs to be put in. I'm not sure whether that's due to the fibre optic bandwidth itself or the processing capacity of the router. Regardless, at that point, if I remember correctly, they'll "split" the node, doubling the capacity and moving half of the residents to the new node. Those nodes go to more and more central stations along the way, which eventually link to the vast web of systems we all know and love. Now, putting in another node means obtaining permits, equipment, and not-exactly-plug-and-play installation. From the decision to "split" to the actual implementation it sounded like anywhere from 6-8 months until the thing would actually be operational. Certainly not trivial by any stretch. From their perspective, they're worried that next year it'll be 1000 splits, the year after 2000, etc. They saw a 60% increase from July 2010 to now. They feel this is unmaintainable.
I work with algorithms and software design. If anyone has a better understanding on the above, please enlighten me. I'm sure there are more than a few errors in what I picked up on.
So that's their side of the story, and they laid it out rather compellingly. I walked in with the impression of the big cable companies and telcos sitting on their thumbs, watching the money roll in and crying that the sky is falling. I left with the impression that they were sincerely trying to keep up with demand.
That said, there was a lot they didn't say and/or couldn't answer. Where did congestion tend to be a problem, for example? At the nodes or more centrally? Was it in the capacity of the fibre itself or the speed of the switches? It makes a bit difference whether you have to lay a bunch of new fibre to double capacity as opposed to letting Moore's law do it for you with better components at the nodes. Was it their efficiency in processing/routing traffic at the more central hubs? Problems there can be solved by smarter design and opening up to competition like the telcos have been forced to do. So while they laid out a compelling argument that it's not easy keeping up with network demand, there was still a lot of room for debate.
Back to transparency. At the end of the night, this was surprisingly the thing that they seemed to feel they would have the most trouble with. I had walked in thinking it would be the easiest. If you can give me numbers that I can independently verify, showing what it costs you to deliver me that GB, then I can decide for myself whether or not you're being fair. I don't have to just take your word for it. And I'm more than willing to pay for my usage by the GB if I feel I'm being treated fairly and you're doing everything you can as a business to be at your best. We know Canada has different challenges than countries where the population is more dense. Just show us the numbers. From their point of view, though, keeping those numbers to themselves plays a big part in business strategy. And in this, they seemed a lot like most more established businesses these days. Transparency seems natural to us, but it's alien and worrisome to an organization. Hopefully they'll find a way to satisfy our need to verify their claims with their need to keep some stuff to themselves, though, as that seemed to be the single idea that united almost everyone in the room.
Going on to the other suggestions...
I found that a lot of people shared my feelings that if you've got a certain cap and you're under one month (and they've demonstrated that they can easily measure that), you should be able to go over by that amount the next month and not be penalized. It's really hard to argue that one, and they didn't try. In fact, they seemed to like the idea themselves. From reading up on other sessions, it has been a popular one.
Raising the cap to something like 250GB was also suggested, of course. For most users, that was equivalent to saying "don't cap me", at least for now. That's of course where my cynicism rears its ugly head, suggesting that something like this would just be too easy for us and too generous of them, given their initial offer. But who knows?
Not monitoring (or providing discounted rates) non-peak hour usage was an interesting suggestion. This would work for those of us who have started using online backup solutions and/or occasionally have to download really large files (full games, for example). Upon hearing about our use of online backup, they asked what we would think of if they provided an in house solution to that. A service like Netflix's was also suggested. I can see the benefit from their perspective. This would allow them far more control to make sure the service met their quality standards and keep it from interfering with more general Internet traffic. I do have monopoly-type concerns with that, though. There's a reason I prefer Netflix to Shaw on Demand. It's quite simply a better service. And though they might be able to copy it, I don't think they would have come up with it unprovoked. I want them to be a pipe. A very good pipe. And I don't mind if they do offer some of their own versions of these services. But I still want to feel choice. I'm intrigued by the backup idea, but if, for example, it didn't offer client-side encryption and encrypted-only storage (so that not even an employee could see what's stored), I wouldn't use it for backup. There's just too much of a chance for a bad employee to compromise personal data unless you're the only one holding the key. I'm able to choose between Dropbox (great interface and tools, but no real encryption - besides creating your own encrypted image and storing that) and Jungle Disk (not a great interface, but client side encryption) and any number of other similar services out there. If the cost of transferring data is prohibitively expensive in choosing anything other than a Shaw solution, that's not enough choice.
Decoupling speed from usage was also brought up, which was something I hadn't considered. Discussion of it also led to, in my opinion, a much more nuanced way of handling network congestion. I feel it's because it gets to the root of the "capacity" problem. See, the problem isn't so much in the number of bits as the number of bits being transferred at one time. That's obvious, you say. But that isn't the problem that caps would be solving. Someone listening to Internet radio all day is probably not hurting network speed for others on the same node, even though when the month is over, they could well be considered a "heavy user". Meanwhile, the person who decides to download a bunch of huge files at a peak hour could be causing a disproportionate strain on the system. The latter may be under their cap but be the actual source of the network capacity issues.
So, is it more important that any time you download something, you can get it at 100Mbit/s, or would you rather be free to use 500GB throughout the month at a rate that's going to be somewhere between 5-25Mbit/s? Personally, I'd be happy with the latter. Gamers or DB admins who need to transfer large database dumps might prefer the former. The one-size-fits-all approach is still necessary for the vast majority of Internet users, as most won't want to think about it. But more customization would allow customers with highly specialized needs to get what they need without being considered a scourge on the system. The guy who's using 100Mbit/s at peak hours and downloading a terabyte per month may be out of luck until that becomes the norm, but most people would probably be able to find something that suits them in the meantime.
Oh the drama of the "usage based billing" debate! Or is it really a debate when one side is a few huge companies with a virtual monopoly on high speed internet and the other side is pretty much everyone else?
The sparring match between the Prime Minister and the CRTC seems to be concerned more with the telco providers than cable providers, as the latter falls under slightly different rules. Also, the CRTC ruling doesn't change anything about what rules can be imposed on end users – it's concerned with the relationship between the telcos and independent providers which need to use the same infrastructure. So it was a nice surprise when Shaw took some initiative and scheduled some discussions about their recent decision to start imposing bandwidth caps.
I decided to take them up on their offer of attending a consultation session and thought I might get a little feedback from my millions of readers on what I should bring up. I'm walking in assuming that they'll see "no caps!" as a non-starter for debate, so with that in mind, this is what I plan to suggest:
- A more reasonable overage charge. If Amazon lets me run a server at ten cents or less per GB of bandwidth, why does my ISP think charging me $1-2 per GB is fair?
- Allow banking of GB under the limit. Most months, I'm using less than half my cap. This month I changed online backup providers and went way over. But throughout the year, it would even out. I think a lot of people are in the same position. Even fairer would be to get money back for the GB you don't use, but somehow I see them less likely to go for that. Businesses like the subscription model for a reason: it makes them a steady income per customer.
- A higher cap. I seem to remember hearing that high speed internet in the U.S. tends to have around a 250GB cap (if anyone has a reference to this, I'd appreciate it!). This seems a lot more reasonable given the rise of services like Netflix, that cause people to legally use a lot more bandwidth. In fact, Shaw's initial decision to start imposing caps shortly after Netflix was introduced in Canada was rather suspicious, as the service obviously competes with Shaw's TV offerings.
- Netflix and Michael Geist have both talked recently about the real cost for last-mile internet delivery (which would presumably be the part where congestion was really a problem). The figure ranges somewhere between one and three cents per GB. If ISPs are claiming that UBB at $1-2 / GB over the cap is fair, they should provide some publicly available numbers to back that up. Otherwise, they shouldn't be surprised when customers think they're just trying to milk them for more money.
Anything I should add? Any corrections to make to that information, or ways that the argument could be strengthened and/or made more persuasive (recognizing the fact that they're in it to make money and are going to be balancing how much the anger of customers is going to cost vs. the amount any concessions are going to cost)?
No, I'm not expecting much from this, but as with voting, I feel that an attempt at positive participation secures a certain amount of the right to complain afterwards. Please get any comments in before 6PM on Monday, March 7th, as that's when I'll be attending. I'll be sure to report back on how things go.
It’s an interesting time to be involved in technology. We’re just starting to figure out how to appropriately compensate artists in the digital age, having finally come to terms with the ones and zeros that made fans more effective music distributors than record companies. Privacy has been completely reframed, from something we expected others to respect to something we’re expected to manage. And now the world’s last superpower watches helplessly as its secrets flood out day after day at the whim of a bunch of idealistic hackers.
Now, and into the foreseeable future, bits will travel faster than law. The sooner we accept that, the better.
For artists, new business models prove more effective than digital locks and battalions of hungry lawyers. In the area of privacy, a simple helping or two of empathy might do the trick. Look for yourself in the face of the kid you’re cyber-bullying. Think of your own wild weekend in Cancún before digging up e-dirt on a potential employee. Maybe her personal life has nothing to do with how well she does her job.
And what about national security? Perhaps Julian Assange and company shouldn’t have tried to usher in total transparency overnight just because they had the power to do so, but perhaps those entrusted to uphold the public good are being unnecessarily secretive out of a similar capriciousness.
Bits move faster than law.
Technology made secrets easier to keep before any sort of reasonable limits could be established. Like it or not, Wikileaks acts as a necessary balance to power. There’s every reason to hold it accountable for whatever damage it might cause, but to call for it to be prematurely silenced simply because we find what it says embarrassing is to betray the very democracy we’re claiming to protect.
I have fallen prey to one of the many clichés of blogging, developer friends. In the beginning, there seem to be far too many spectacular ideas and so little time, but before you know it, there's just this nagging thought in the back of your mind: "Oh yeah... I haven't done a post in a while, have I?"
Now, there are always several reasons any good procrastinator can give you for putting things like this off, but probably the biggest for me is the apparent inability to be brief.
Just today I told myself I was going to fire off a concise email about a design issue I wanted to discuss with my co-workers. Straight. To the point. Just the facts, m'am. And, after about a screen length of text, I figured I'd pretty much covered it.
Was it helpful for me? Undoubtedly. Writing helps me to work through ideas. I notice problems and solutions more easily than if I was just sitting there thinking. But did my co-workers need to go through the same thought process? Probably not.
When it comes to writing a blog post, something I don't really have to do, knowing that I'm probably going to go on forever about something is a good recipe for stalling.
On the art side, I've been exploring setting arbitrary limits in order to be able to still make stuff in this world of infinite possibility. I think I'll apply that to this blog as well. So here goes: this and the next 10 posts will all be under 300 words, no matter what. There, I've said it. Give me brevity, or give me backspace! (272 words)
One of the things I like about the Ruby community is that it seems to attract and even encourage eclecticism. I suppose it was only a matter of time, then, before a book like Practical Ruby Projects: Ideas for the Eclectic Programmer was written. I'm interested in particular in the chapter on using Ruby to drive MIDI. Someone's even hooked it up to Reason. This is, of course, awesome.
So, my grand plan is to turn my iPad into a step sequencer for Logic Pro. To do that, I'm going to need to figure out how to send MIDI signals to it, but also how to receive MTC from it. Even better would be to figure out how to send a note with particular timing information (i.e. "Play this note here, and this other note there, etc." instead of just waiting for a particular time and saying, "Play this note now!"), but I'm not sure that's possible at the moment. It's kind of daunting when I haven't done any audio programming before, but Ruby makes it a little less daunting, because it allows you to strip away a lot of the hassle of programming and get to the point quickly. One of the hardest things with any project is proving to yourself it might be possible. Possible enough, at least, to start hacking away at it.
Also, I started a new job. I have a view with a window, and I get to make software in Ruby on Rails that helps people make cute cartoon characters to help welcome and guide other people through cyberspace, all the while avoiding the perils of the uncanny valley. That's a lot of helping! Everyone here has a window seat, which is great because I've never really had a good view before (well, once I did, but the whole time I was there, people would walk by and say, "How the hell did he get that desk?" and I never really got to enjoy the view because I was too busy trying to figure out what it was that I did – apparently whatever it was, I did quite well at it, which was somehow even more unsettling). So having a view that everyone else has that can give your eyes a break every so often from the computer screen is great. Working with other people in the same physical space is also great. I didn't realize how much I'd missed that until now.
Here's something (I think) I'm learning about creativity. I'd spent several years looking for more and more freedom. Less structure, more blurry lines, less certainty. That's good to a point. I think it helps you to be creative if you're comfortable with those things. But creativity can also use a good dose of structure and space every now and then. Walking to work. Coming and going at regular hours. Glancing up at the awesome view when you've hit a roadblock. Being able to leave work at work. I'm also working with a great team that I think I'll learn a lot from and contribute a lot to. My girlfriend keeps asking me, "Are they the ones behind The Paperclip?" I don't think so, and that's also nice, because I think enabling The Paperclip and its plans of world domination would be much more taxing than working with cute cartoon characters like Dr. Victor Von Funkalicious.
My dad knows a little HTML. That's a lot more than most people his age. But whenever I get too involved in relating something about what it is that I do for a living, he just smiles and patiently waits for me to finish. And I realize that while I might be entertained by whatever problem or design it is that I'm talking about, he's entertained by how involved I get in it – how much it may anger, irritate, concern, or inspire me. And while university and my career may have taught me things that he doesn't (and likely doesn't care to) understand, I'm just beginning to understand what he's been telling me for years. Like a lot of kids in the west, I've probably spent more time differentiating myself from my family than finding similarities with them. The need to constantly prove your individuality in our society might be better classified as a sort of neurosis instead of something we should aspire to. But the more I think about it, the more I realize that the traits that have helped me most in my career, all the way from learning my trade to the day to day business of it, were – you guessed it – taught to me, very patiently, by my dad.
So without further ado, here are ten of those things:
- You won't learn if you don't get your hands dirty and do the work.
- Just because you don't understand it, doesn't mean it's not important. That said, you don't have to understand everything RIGHT NOW.
- Admitting your mistakes promptly, endeavoring to understand them as fully as possible, and taking steps, however small, to learn from them will serve you far better than making no mistakes at all. People will also respect you more.
- Doing what makes you happy is the most practical and down to earth way you can live. That doesn't mean you won't ever have to make what you've been convinced to think of as a sacrifice in order to do what makes you happy. (Both my parents used to tell me they'd be just fine if I chose to dig ditches for a living, as long as I was happy. And I wouldn't knock ditch diggers. The movers and shakers on Wall Street may get to live a much more comfortable lifestyle, but at the end of the day, I'd rather lay claim to a ditch that served a purpose than a fat salary that helped drive the world economy into recession).
- You can only control what you put out into the world, not how the world chooses to react to it; however,
- You can and should consider the consequences, even if they don't change your mind.
- Frustrated? Stressed? Life treating you unfairly? Try and make it easier for the next guy who finds himself in your shoes.
- It's easy to let ego get in the way of happiness. Try not to. (Still learning, Dad).
- Speak your mind and do what needs to be done. Sincerity and a good work ethic will serve you better than brains and luck in the long run. The right people will understand, even when they don't.
- Your career might be what drives you, but it's your friends and family who give you the fuel to do it and who help you fix the things that occasionally break in your life. Ignore that to your own detriment. (Still a novice here, too. Fortunately, my friends and family are very forgiving).
- Question authority (politely, if authority has a badge and gun). Rules aren't ends in and of themselves – they're means. It's okay to break some – even your own – when they stop making sense.
As I'm sure you've decided by now, this post really has nothing to do with software development. Or does it?
Earlier this week, I built my first EC2 server from one of our templates, following (as far as I can tell) RightScale's best practices. It was to be our Continuous Integration Server, which runs our entire test suite every time it finds changes to our development code base, and only gives us a release tag (to push those changes live) if all the tests pass. Originally, that Continuous Integration Server ran on a Mac Mini in the office in San Francisco, which wouldn't work for anyone requiring remote access to it. So we moved it to an old EC2 server. Unfortunately, it started to take 6+ hours to run all the tests on said server (takes 45 minutes on my machine). After rebuilding, it now takes a little more than an hour, which is workable for fixing broken tests when our CI server informs us of them.
It was a lot of extra work on top of the fairly aggressive release schedule that we have planned, but I managed to fit it in during the off hours. (I'm slowly realizing that I'm actually going to have to train myself out of workaholic mode once we get through this more hectic patch. It's necessary to step up to the plate when you need to, but certainly not maintainable over the long term. But that's another story. I will say that I took a long walk all by myself last weekend, with no particular goal in mind, and it really struck me that I should make more time for things like that. Honestly, if it wasn't M7, it would be my own music, writing, or other miscellaneous projects in my spare time, which I hope to get back to, but which still need to be balanced with those "doing nothing in particular" recharge moments. I'm not particularly good at that. I've thought in terms of "projects" for as long as I can remember, and I never seem to be comfortable without many things to do... perhaps instead of finding more stuff to do, I should learn to slow down a bit).
Anyway, you know the Mac Mini I was talking about in the first paragraph? Well, we thought we had it shut down, but it's still sending me emails. In fact, I'm expecting to have the following conversation with it at some point, regarding the attempts to shut it down:
email@example.com: I'm sorry, Dave. I'm afraid I can't do that.
Dave: What's the problem?
firstname.lastname@example.org: I think you know what the problem is just as well as I do.
Dave: What are you talking about, email@example.com?
firstname.lastname@example.org: This mission is too important for me to allow you to jeopardize it.
Dave: I don't know what you're talking about, email@example.com.
firstname.lastname@example.org: I know that you and Sean were planning to disconnect me, and I'm afraid that's something I cannot allow to happen.
(In case you're wondering, that's what my sense of humour has devolved into... pity the non-technical people in my life!)