Social data flow in action

OK, I’ve seen a few things online recently that amused me and I’d like to share them. I don’t want to add them to delicious.com because that’s what I use for links I need to find again someday. These are things I’ve seen and enjoyed but they are gone. For me they are yesterday’s newspaper. But you may not have seen them and I think you should.

So I’ve decided to add them to Tumblr. Easy. Tumblr is a great tool for doing this, it makes it easy to add videos, links, pictures, anything. I am very happy with this choice.

But this has now repurposed Tumblr in my social data flow from an output device to a content provider. Before, it just used to receive my status updates and micro blogs. Now it is the place where I link to things that have amused me.

Here’s how I needed to reconfigure my social data flow:

1. Add Tumblr to my aggregators. Now my Tumblr posts will appear in my All Activity feed.

2. Remove Tumblr from my syndication list. Now Tumblr will not receive my status updates or micro blogs. This is important because otherwise these updates would appear twice in my All Activity feed.

3. Er…

4. That’s it.

Simple.

Pedestrians and velocity

John Whiteside made a perceptive comment about my post “Cycling to work - why people hate cyclists, part 3“. I completely agree with him - what he said in his comment is what I intended to say in my post. Sorry it wasn’t clear.

The post talked about collisions between cyclists and pedestrians. I said

The cyclist will see a pedestrian many metres ahead and will calculate his own trajectory and the pedestrian’s to ensure there is no collision.

John correctly commented that pedestrians don’t have a “trajectory”. What they have is a “radius of occupation” - a theoretical circle around them that they could be anywhere in when the cyclist reaches them. The radius of the circle is a function of the pedestrian’s maximum velocity and the distance the cyclist has to travel before impinging on the pedestrian.

Here’s what I meant to say: the cyclist unconsciously calculates this “radius of occupation” and avoids it.

Of course, there are some parts of the circle that the pedestrian is unlikely to occupy. The sophisticated cyclist might try to calculate the probability distribution across the entire “radius of occupation” and choose to clip a part of the circle where the pedestrian is unlikely to end up. This sort of calcuation will be correct most of the time, but ultimately will result in a few collisions.

I think such collisions are the fault of the cyclist. My diatribe against pedestrians was intended to include only those collisions that the cyclist could not possibly avoid.

How organisations really work

Benjamin Waber has a grim piece of news for managers and CEOs: You’re out of the loop.
- Wired magazine, 16 August 2008

This is something we all suspect about large organisations - unless you’re a senior manager in which case you probably think all valuable communication goes up and down the management hierarchy. Clive Thompson’s short article in Wired makes me want to read the full MIT paper by Benjamin Waber and his colleagues (how can I get a copy?).

Almost every time he analyzes a group, Waber discovers that the super-connector — the crucial person who routes news among team members — isn’t the manager.
- ibid.

The research team discovered this by putting location-aware badges on everybody and mapping how long they spent in proximity to each other. Using this information they could identify the Jan Molbys of the organisation - the midfielders who made and received the most passes. Information was flowing through the organisation in ad hoc patterns, enabled by these super-connectors. The data was good enough to predict when and where some of these exchanges would take place in the future - information that could be used to make sure the super-connectors were able to do their unofficial function more effectively.

I have worked in organisations where this data would be used to quash this sort of unofficial communication. Where is the cost-benefit analysis, managers would cry? How do we know these people aren’t simply gossiping? How can we keep them at their desks longer?

This is why the permanent use of such location-aware devices isn’t going to happen in any organisation I know. Such snooping would certainly be verboten by the Workers’ Council in any German organisation. For every benefit to the organisation and its employees there is a potential Big Brother use too - what’s to stop the people in possession of the data from identifying people who spend too much time in the restrooms for example. There’s a trust issue, and if you’re in an organisation where the senior management is trusted then its communication channels are probably in a fairly healthy state too.

So the permanent use of location-aware badges is unlikely to happen in your organisation, but the knowledge gained from this experiment could be of lasting value. It might be possible to identify the super-connectors in your organisation without such intrusive monitoring. If you value their networking skills then you should find ways to support them.

Revised social data flow

In an attempt to remove the latency problem from my social data flow, I’ve rearchitected it a bit. Unfortunately the reworked flow is not quite as elegant as the original but then it had more sophisticated design goals so that’s not terribly surprising.

Here’s the revised flow with notes below:

(Click on the image for a bigger version)

Notes:

1. Now I can allow ping.fm to update Twitter and identi.ca immediately so I have removed the latency problem from the previous configuration. Once again I can use Twitter in a conversational way instead of it being a mere receptacle for RSS updates every half hour.
2. You’ll note also that Twitterfeed can now update identi.ca and other Laconica-based micro-blogging sites. Thanks, Mario. This means I can promote identi.ca to be a peer with Twitter and receive all my updates including real-time micro-blogs (tweets).
3. The three subscription points are the same: Writing, Status Updates and All Activity. The URIs for those resources have not changed.
4. I have distinguished between blog posts such as this and the notification of those posts. To subscribe to the full post stream use the Writing feed. If you subscribe to the All Activity feed you will be notified about posts but only see the title not the whole article. Unfortunately if you subscribe to both you will see both the blog post and a separate notification of its existence - I can’t do anything about this yet.

So there’s an emerging gap between those select sites that I use to do stuff and those sites that merely receive notification of that activity. My strategy will be to keep to a minimum the sites I use for doing things, and maximise the number of sites that are notified about it. This way I can keep all my pictures, links, writing etc. in one place but all my disparate multiple friend communities will get equal notification of new activity in one of those upstream sites.

I think the new slightly less elegant social data flow is functionally better and I’ll try to stick with it for a while to see how it performs in practice.

Initial thoughts on my new social data flow

Firstly, apologies to anybody who couldn’t see the social data flow diagram in my earlier post - I’ve now hosted it on my own server so it shouldn’t disappear quite so often.

(Click on the image for a bigger version)

The new data flow, using ping.fm to syndicate my status updates and micro-blogs and FriendFeed to aggregate my other activity and consolidate everything into a coherent set of feeds, is working quite well. I would like to be able to have more control over the format of what both ping.fm and FriendFeed do for each target system, but the defaults are adequate for now.

Here are the issues I’m experiencing:

1. Using Twitterfeed to take my FriendFeed RSS and pipe it into Twitter introduces a degree of latency that’s a bit inappropriate for Twitter. Twitter should be near real-time and conversational (whilst not being an instant messaging system), but Twitterfeed only reads my FriendFeed every half an hour. If I get ping.fm to update Twitter directly then I’m going to get some updates twice, which is exactly what I’m trying to avoid.

2. Some of the connected systems (identi.ca for example) are Twitter-like, and I’d like to treat them the same way as Twitter: as an output device for all my activity. But Twitterfeed is Twitter-specific and I haven’t found an equivalent for identi.ca etc. yet. In any case I don’t want a different Social ETL tool for each output system; I want Twitterfeed (or something) to do this for a number of target systems.

3. There’s no class solution for location-based systems yet. Obviously I’d like to be able to update my location and travel plans in one place and have it propagate to TripIt, Dopplr, BrightKite etc. I think FireEagle may be trying to do this but it’s short of two things at the moment: co-operation from the other systems and an invitation for me. If anybody has access to FireEagle I would appreciate a way in.

4. Now I need to do the other thing - deduplicate my profligate friends’ updates. When Myrto uploads a picture to Flickr, for instance, I get notified about it four times. I can rationalise this a bit but I can’t ignore her FriendFeed or Twitter because I would miss some of her updates, so I’m condemned to hearing about her new pictures multiple times unless (a) she adopts a version of my social data flow or (b) I find a solution to deduplicating the same update coming through multiple channels. This duplicate filter is necessary until everybody adopts my architecture, i.e. for ever. Does anybody know of a solution out there?

This all seems quite difficult to manage and well beyond anybody who is either busy or technically challenged. There’s no real way of developing a packaged solution until all the social systems adopt a single sign-on technology like OpenID, and many of them have sound technical reasons for not doing so (OpenIDs can be created by anybody anywhere so it’s a bit like expecting system owners to trust a digital certificate that has no trusted root certification authority).

Anyway, that’s the state of play at the moment. More to come I’m sure.

My social data flow

I got in a bit of a tangle with what Web 2.0 application was updating what so I’ve tried to rationalise the data flow like this:

(Click on the image for a bigger version)

Having done this I’m fairly sure of three things:

1. There are no feedback loops: no update is going to propagate to another site that feeds it back where it came from
2. There is a specific feed for three appropriate levels of details that does not omit anything relevant.
3. There is no need for any other feed unless you’re interested in particular asset classes, e.g. photos.

There is one publisher, one syndicator and one aggregator. Each of them can be replaced at any time without affecting the principle of how this works.

So here is the guide to subscribing to updates about me:

Give me everything!
I love hearing about what you say and do and I can’t get enough of you
OK: subscribe to this feed

I want to read what you write about
I’m not interested in where you are or what you’re doing today but I am interested in your thoughts
OK: subscribe to this feed. And to this feed for what people are saying about my writing

I’m interested in keeping track of you but I don’t want to read your boring writing
I’m a stalker not an bleeding intellectual
OK: subscribe to this feed.

Hopefully that will cover everybody. If you’re interested in specific stuff like photos then go to my FriendFeed page and choose the source you want.

Notes on how I rationalised this:
1. I now use Twitterfeed simply as a mechanism for posting RSS to Twitter, specifically my FriendFeed stream. I’ve classed it as “Social ETL” in the diagram

2. Twitter is now just an output device and a way of feeding my updates to mobile phones, for example.

3. I have tried to remove all Facebook apps that update my status except for ping.fm

4. Sorry for the multiple and egregious posts while I sorted this out today

5. Thanks to Myrto for the ping.fm invite.

Trying to recover my compromised Hotmail account

[Edited to add: Microsoft Live support resolved this issue very swiftly and professionally. I now have control of my Hotmail account back.]

Somebody, let us call him Anthony Ikpe, has hijacked my Hotmail account and sent a begging letter to everybody in my address book. A transparent ruse but apologies to everybody who got this message. Needless to say I am not stuck in West Africa nor do I need an urgent transfer of funds to help get me home. If you would like to send me money then please do so, but don’t send it to an address in Nigeria.

I don’t know how Anthony Ikpe was able to get access to my account, but he has managed to change the password, change the alternate email address and change the challenge-response questions so I cannot get back control of the account by the usual means. My old password was not a strong one, but nor was it an easily-guessable one. I suspect he has somehow spoofed the password reset system, which would be a security issue for Microsoft if it is true.

My Hotmail address is the one I publish publicly - the Hotmail account is designed to be the one that gets attacked. My friends, family and colleagues know that it’s not the address I normally use. I also use it for buying things online so that the resulting marketing messages don’t bother me. However, if I’m in the middle of a buying something it assumes a temporary importance. And I am indeed using my Hotmail account quite heavily at the moment. Or not, as it turns out.

Inconvenient then, and I’d like to get it back if possible. And I’d certainly like to stop Mr Ikpe from reading the messages I’m currently getting on that account. So I am trying to contact Microsoft to let them know what has happened and see if anything can be done.

Attempt 1

I click on the Help link on the Hotmail home page. Yay! there’s a link for “What do I do if I think someone is using my Windows Live ID? “. I click it. It gives me another link to “Report to Microsoft” any suspicious activity.

I click it. It leads to a Technet page with another link telling me to fill in a form on this page: Helpful Information for Microsoft Passport Network

I click the link. It takes me back to the Hotmail help page.

Presumably Microsoft think their customers are so stupid they will continue round this loop for ever without noticing they aren’t getting anywhere.

Attempt 2

I call Microsoft UK. After patiently navigating their call centre menus, I arrive at the Hotmail options. For billing enquiries please press 1. For all other enquiries please go to http://uk.msn.com

My first reaction is Gaaah! But surprisingly there is a form on the MSN site that allows you to report Hotmail security issues. I filled in the form and submitted it. The waiting begins.

A thought experiment concerning the Laws of Cricket

This kernel for this post was the reaction to Kevin Pietersen’s “switch hitting” exploits in the recent One Day International against New Zealand, but this post is not about switch hitting as such. The thought experiment was suggested to me by a friend (and now team mate) Graham Pontin and it is an interesting one.

The furore caused an examination of Laws 36 (Leg Before Wicket) and Law 41 (The fielder). Law 36 says the Leg Side is where the batsman’s legs are when the ball comes into play (at the start of the bowler’s run up). Law 41 says that there shall be no more than two fielders behind square on the leg side when the bowler bowls the ball.

Here’s the thought experiment: imagine I am a well-known international right-handed batsman in a Test Match (I said it was a thought experiment - use your imagination!). The field is set - there are two slips and a gully. At the start of the bowler’s run up I take a left-handed stance. During the run-up I change round to my normal right-handed stance and play a normal shot.

If the slips had lined up for my initial left-handed stance then I have taken them out of the game by turning round; it’s a free hit. If the slips had lined up for a right-hander then it would be a No Ball on account of them being behind square on the “leg side”. If I am struck on the pad by a ball pitching outside what is effectively my off stump, it’s Not Out on account of the Law saying it’s actually my leg stump.

It seems like I am getting a clear advantage by lining up the wrong way then turning round as the bowler approaches.

The question is: what should the bowler do? Or perhaps, being a bowler and therefore somewhat dim, what should his captain tell him to do?

1. Bowl outside the (right hander’s) off stump
Bad tactic because there are no slips or gully.

2. Refuse to bowl until I turn round
There are big fines for slow over rates these days. You’ve got to bowl sometime.

3. Bowl wide outside the (right hander’s) leg stump
This seems to be the most promising tactic - the umpire ought not to call a Wide since it is legally still the off stump. However, this is stalemate - you can’t get me out like that. And I could still improvise a shot if I know this is the tactic you have chosen.

Does the Law need to be changed? I think not - there’s always Law 42 (Fair and unfair play). As an umpire I would be quite happy to say that the imaginary batsman is not acting within the spirit and traditions of the game, and to impose the sanctions described in law 42.10 (Batsman wasting time) - one warning to the batsman then for a second offense five runs to the fielding side. I didn’t actually know this sanction existed - the runs are added to the fielding side’s previous innings. Does anybody know if this has ever happened?

This is not to stifle innovation - Kevin Pietersen’s exploits are admirable and an adornment to the game. But a batsman attempting to exploit the Laws in a legalistic way is certainly wasting everybody’s time and should be penalised.

There’s your answer, Graham.

Birdsong

Years ago, when Classic FM was first starting up, they did some test transmissions on the new frequency. I happened to tune in one day while driving. The test transmission was simply a recording of birdsong in woodland. Wow. I didn’t listen to another radio station in my car until the proper Classic FM programming started. Obviously I tuned away pretty darn quick at that point. Henry Kelly. Enough said.

Fiddling with a DAB radio at home yesterday I happened upon a digital station called Birdsong. Yay! It’s the same recording I think. I cannot recommend this station highly enough. It’s incredibly calming to hear the sounds of the countryside when you’re sitting in an office surrounded by technology.

Do try it.

Am I a man?

	Yes/no
I can keep my head when all about me Are losing theirs and blaming it on me	Yes
I can trust myself when all men doubt me, But make allowance for their doubting too	Yes
I can wait and not be tired by waiting	No
Being lied about,  I don’t deal in lies	Yes
Being hated, I don’t give way to hating	Yes
I don’t look too good, nor talk too wise	Yes
I can dream—and not make dreams my master	Yes
I can think—and not make thoughts my aim	Yes
I can meet with Triumph and Disaster And treat those two imposters just the same	Yes
I can bear to hear the truth I’ve spoken Twisted by knaves to make a trap for fools	No
I can watch the things I gave my life to, broken, And stoop and build ’em up with worn-out tools	Yes
I can make one heap of all my winnings And risk it on one turn of pitch-and-toss, And lose, and start again at my beginnings And never breathe a word about my loss	Yes
I can force my heart and nerve and sinew To serve my turn long after they are gone, And so hold on when there is nothing in me Except the Will which says to them: “Hold on!”	No
I can talk with crowds and keep my virtue	Yes
I can walk with Kings—nor lose the common touch	Yes
Neither foes nor loving friends can hurt me	No
All men count with me, but none too much	Yes
I can fill the unforgiving minute With sixty seconds’ worth of distance run	No
I am 72% Man

It’s possible Kipling meant this to be a binary choice. In which case I am clearly not a man, nor is the world and everything that’s in it mine.

Here’s a handy Manliness calculator for you.