Parallax Effects: Mutable Development and Modern Map Moving

yelp-map-mover-bubbleUntil a months ago, Yelp featured a modal pop-up when users tried to click-drag the slippy map provided to situate search results according to a local topography. The prompt was “Hey Map Mover?…blahblah

merrytimes art project

Screenshot of MerryTimes, a parallax project at the Holocenter, NYC

My parallax projection project with Julian Burgess (say it, 5x fast) was not as intense or playful, but probably on the spectrum between those last two, it’s some solid hyperlocal-fun about fuzzy riverscapes and insular environments. The general objective of parallax visuals, ZUI interfaces, and anything that implies movement on a 2D screen is to immerse you in an environment or a static scene and draw you into the automagic of its motion. Constant motion, “agile” development, characterizes a lot of technology and journalism of late as well, and parallax has become a pretty popular and loosely applied concept in the realm of online interactives and visual data re:presentations.

Movement and the Making of Open Maps

But back to how this relates to my current condition and career path plan. I think about parallax and movement a lot when I describe the agility and adeptness that my job requires, and the kind slurry of opportunities the Open News Fellowship provides.

On the daily, I juggle lots of things working for two software and data-driven journalism programs (Ushahidi and Internews-Kenya). My coding projects range from contributing to open source citizen journalism and crowdsourcing platforms like Ushahidi Version 3.0, or 30 year retrospective interactives studying the semantics of socio-political perspective on East African health, or providing visualizations of violence report data as part of election monitoring initiatives in Yemen and Nigeria, or plotting environmental sensor data from collection stations across Tanzania in a web-based viz series. As asides, I develop courses and curriculum for a non-profit that I help run in NYC called Girl Develop It and mentoring budding coders at the Academy for Software Engineering in New York. Tacking on the weekly radio show, and occasional art projects; I accumulate an impressively deep collection of random.

Point being, I learn constantly, work assiduously and have the opportunity to meet at conferences and in collab code sessions with some of the most impressive, genuine and amazing humans in the tech/journalism space. In my short (now 6 month-ish) tenure as a fellow, I’ve traveled to five continents, lived on two, attended over 10 conferences, spoken at 8 of them, built open source projects with collaborators at 5 of them. You can check my commit history on github or my fellowship repo where I’ve been tracking projects and speaking engagements, including my now rather abortive attempt to convert github issues into a todo list manager for my all of my talks and tasks. For now, I’ve been logging my ongoing projects, conference notes, and both present and potential work thereabouts, and so the version-controlled storage model provides pretty fair understanding of the constant change, abrupt but amazing shifts, prolific blogposting, and pretty persistent parallaxes of my fellowship program.

And not to be too simultaneously overly-alliterative and Cindy-Lou-Who-twee with the parallax refs, but I think some of the most important developments of this experience haven’t been logged in a blog post or my tech-tonic shifts of my task documentation; some of the most important developments have been in a personal discipline I’ve mustered through these experiences, to manage ALL THE THINGS, prioritize my own workflows and organize my life to more maximal productivity. Part of the elegance of the parallax metaphor in this case is in it’s homonymous “pa-relax,” which can maybe best reflect in a portmanteau that some of the most sustainable lessons are those that allow you to partner the extreme opportunities and impressive responsibilities, with an ability to manage expectations methodically, and support your collaborators without stress or burnout.

This requires some flexibility, some skill, some patience with constant change, some humble understanding of your own personal limitations and a healthy refusal to be stunted by them; these are qualities I’ve developed more thoroughly in recent months, and recognized pretty profoundly in my fellow fellows. They’re qualities that everyone should seek independent of their embrace of a parallax life model or map moving philosophy. If you want to do great things, develop professionally, collaborate actively, progress personally, and contribute productively to a global or even niche-local community, these are qualities you’ll probably invest time in developing. Why not do so with some support along the way?testing the parallax art app in situ

Progressive and Open (Source) News

To that end, you should probably apply for a fellowship like mine, though your experience won’t be Holocenter wall textidentical. An Open News fellowship is simultaneously the most flattering, exhausting, and exhilarating honor, and jobs after it will be hard to reconcile with the epicness of their precedent. My goals prior to the fellowship were pretty basic: code more in the open, learn more about design patterns and backend development, develop sustainable architectural habits instead of one-off project drill behaviors. The scope of my current learning and side-project -> tech stack spread is pretty impressively expansive compared to my initial projections.

Some of the more tacit benefits are nearly impossible to articulate without being gushy. It’s the stranger famery you’ll experience in the news community that clashes with your impulse to imposter syndrome; the kind where you’ll get requests to collaborate on projects from strangers instead of just your friends. Pre-fellowship, I never really had comments on my Github projects and my public code persona was pretty weak; 5 months in, I get regular email about blog posts I’ve written and repos I’ve open-sourced. I speak at conferences where people preface their questions to me with “I read/looked at your…”, instead of just “hey so I disagree and can prove I’m smart by asking an un-question.” (We’ve all witnessed these people; I witness them less now).

Ushahidi V3: Soon with more maps + graphs!

Ushahidi V3: Soon with more maps + graphs!

parallax-brck

my brck, ready to beta-test

As an added perk, I get to work with people who develop technology solutions for under-tech-privileged parts of the world, places where access to the internet might require cell network fail-over solutions like BRCK, or where tracking citizen opinion via social media streams aggregated and API-ified in CrisisNET is so epically important because these opinions are routinely throttled by oppressive or inegalitarian agendas. I beta-test and help build the budding work that will change how we process open information and how we crowdsource commentary. I work with people who train journalists on the ground in nations where objective journalism can be unwelcome, and the practice of building a data journalism program is as-yet nascent or continuously undercover. I’ve had the opportunity to fuse collaborations with the brilliant trainers and journalists at Internews and the smart and savvy technologists at Ushahidi; and in this capacity, I work with and support people who develop new ways to provide critical perspective on sustainable sociology and global health programs, on geopolitics in the absence of privacy and in the face of human rights violations.

I think about this more and more in the context of how journalists have adapted to suit these demands, becoming multi-tasking reporters/technologists/domain-specialists/data-“scientists”/activists. A recent BrainPickings article about Achebe’s approach to writing as a vocation seemed to fit here too, where “writer” adopts all of these newfound technical roles while maintaining the same ethical mandates, explaining what Maria Popova so awesomely cited as the “redemptive power of fiction and Chinua Achebe’s perspective on optimism in lieu of ‘clickbait’ journalism”:

[My idealism is] still alive and well because without it the business of the writer would be meaningless. I don’t think the world needs to be told stories of despair; there is enough despair as it is without anyone adding to it. If we have any role at all, I think it’s the role of optimism, not blind or stupid optimism, but the kind which is meaningful, one that is rather close to that notion of the world which is not perfect, but which can be improved. In other words, we don’t just sit and hope that things will work out; we have a role to play to make that come about. That seems to me to be the reason for the existence of the writer.

Hologram Glasses at the Holocenter

Hologram Glasses at the Holocenter

My work is fulfilling, optimistic, open, and ever-changing. Yours could/should/would be too.

Don’t wait
August 16 is the deadline date!

(; I drop rhymes like this errryday ;)

Read all about it here | Apply here

Ping me for any questions here: aurelia[at]ushahidi.com

NYPL Labs also has a map-makers residency for interactive artists, if that’s more your bag

Tagged , ,

Op-interest: On Opinions + OpSec

On of my Webster Words-of-the-Day this week was “opine,” the act of having and stating an opinion. It’s something that I do often on my blog, but am encouraged to stem for a more objective perspective when it comes to professional data vis stuffs and news publications. Journalism solicits an ideally balanced representation of information, but as with any domain touched by human fallibility, it’s vulnerable to bias.

asikstaliban-def

 

The Swedish have an interesting word for one-sided opinions: Åsiktstaliban, defining a group of people who tolerate only one opinion and can be colloquially synonomized with and global violence; and so this blog post is going to address opinion diversity and operational security, two poles of a global approach to citizen journalism and political activism. It seems an appropriate post in the days following September 11th and the tragic anniversary of the Westgate mall attacks, and definitely something that has peaked my [op]interest with as yet feeble articulation these past few months.

Like many developer-journos, I’ve been following the more tragic and graphic media reports out of Iraq, Gaza, and Syria lately. Jonathon, one of our developers at Ushahidi and Chris, his partner on the CrisisNET project, created a timeline of ISIS happenings a few weeks ago, followed by subsequent investigations of conflict in Iraq and Gaza, and this had me reading more about security and media verification for journalists in the Middle East, and otherwise hostile-to-media and humanity areas.

I touched on these topics briefly during my panel at HOPE-X with Harlo Holmes and Barton Gellman (livestream here), and again during our workshop on opsec last week and the Buenos Aires Hacks/Hackers Conference.

opsec2-paneltweet

But independent of my own stuff, there’s a recent trend in crowdsourced citizen journalism that I want to encourage and support professionally and just personally. Part of supporting that initiative is providing open source tools to enable citizen reporters (like those in Ushahidi’s Toolbox), but part of it is also just sharing information openly about authoritative sources.

crisis-response

This is a good place to promote Bellingcat, and other work aimed at armoring activists, newsies, and the general public with information. While it probably won’t keep extremists from more barbarous and cowardly expressions of violence, being informed is non-trivial in the fight against global rights violention. A lack of information historically and consistently is the root of epic geopolitical blunders, tragic massacres, ignorance and ignoring of massive human rights transgressions, globally. To that end, and in a modest objection to the wave Åsiktstaliban media, I’ve assembled a small collection of links and sources to keep apprised of what is happening in places that are remote from my current locale. I’d love to solicit others so I’ve made a form at the bottom of this blog for collecting relevant media sources and tracking the safety of embedded journalists in the Middle East.

  • The New York Times had pretty decent coverage, McClatchy’s wires on the Middle East and the Guardian’s Liveblog have been pretty consistently informative
  • On twitter, I follow Blogs of War (@blogsofwar), and some specific journalists embedded in regions of interest (@BklynMiddleton, @IvanCNN, @Matthew__Barber, @Mudar_Zahran, @jrug, @abumuqawama, @joshuafoust,@combatjourno,@SajadJiyad,@RaquelEvita,@DrZuhdiJasser,@majidrafizadeh,@Reem_Abdellatif,@WalidShoebat)
  • I’ve started reading local bloggers and certainly Bellingcat
  • Vox had a pretty o.k. abbreviated breakdown of the current affairs vis-à-vis ISIS, HuffPo has a decent world roundup as well

But despite the intense media house coverage, I find myself often returning to individual blogs and the work of lone journalists; I think this trend is significant and I’m sure shared by many given the popular response to citizen journo-projects like Bellingcat. I find most embedded journalistss and local citizens to be the most informative for thorough and unapologetically blunt coverage.

israel

 

wtf

As a personal/pseudo-professional aside, we’ve (@Ushahidi) also been working on an implementation of some data visualizations for election monitoring in Yemen, and this had me researching more of the political climate there (so samples below).

prototypes

preview of Ushahidi V3 Viz

I’ve been wanting to build a visualization of global disappeared populations, of which there are many, in almost every country. Those that we hear about more often harken back to Colombia and Argentina circa the 1970s persistently through today, or more recently the 600+ Nigerian girls kidnapped by Boko Haram, the Yazidi women kidnapped by IS affiliates, or the Zone 9 Ethiopian journalists still detained in East Africa. When a country succumbs to brutal regime rule, it’s often the journalists, the vocal activists, and the outspoken citizenry spreading independent opinion and information about injustice that become the targets of violence and effacement tactics. Information becomes a target, and those who process and disseminate it are vulnerable to attack.

soltoff

Screen Shot 2014-09-18 at 2.35.38 PM preview of Ushahidi V3 Viz – gender counts

Perhaps some of their anecdotes and needs are things we might accommodate in the newer version of Ushahidi, or in CrisisNET, our pretty rad aggregator of social and streaming data on the global crisis situation, unified in a single API. And while there are many visualizations and representations of the statistics around targeted terrorist groups, a direct comparison between the composition of the victim population and the terrorist perpetrators is something perhaps worth investigating. A recent open analysis on government documents about outstanding terrorist threats and the TIDE “watchlist” (see also TIME, The Atlantic) reveals some interesting statistics about the paucity of females associated with violence as terrorists, but the general density of females associated with violence as victims.

TIDE by the Numbers

watchlist-by-gender2 of the 9 detained Ethiopian journalists were women, 600+ of the Nigerian girls where; a substantial portion of the limited documentation on Syrian disappeared citizens catalog female adults and children, and coupled with the female rights violations in Yemen, the disappeared counts are also substantial; the same goes for Turkey and countless other nations who’ve only begun to catalog disappearances publically. The Support Yemen Project has done some work to publicize the circumstances surrounding human rights and free expression throughout Yemen as has the AHA Foundation, in addition to logging the impact of terrorism and counter-terrorism, and yet much of my more informed perspective on women’s issues and violence in Yemen stems from the posts of a Yemenese female blogger. Again, my focus returns to local journalists independent of media affiliation. And while females are not the sole-authors covering female rights, the dangers faced by female journalists in terror zones corroborated with some recent reports from the NYTimes and the Huffington Post, as well as some more general blogposts on women’s rights violations authored by the aformentioned lone-journos I follow on Twitter. The circumstances demand a more responsible way to monitor and vet on-the-ground activity and reports, and increasingly social media monitoring and crowdsourcing applications are providing these windows to supplement the occasional blog post and media supported piece.

Screen Shot 2014-08-11 at 11.58.22 AM

This made me consider the unknown, and absence of information as an important root to some of the more brutal disappearances, and particularly lead me to consider the position of citizen-journalists who seek to amplify information about a space and are subsequently pushed by kidnapping, eradication, or imprisonment and public execution. Information is sometimes the most dangerous currency to smuggle from a vacuum, it can mobilize nations to send aid or commence peace talks, it can prompt the vicious reactions of groups who would execute victims to deter action, it can push citizens to technological circumvention tools in an effort to counter the habitual throttling of their internet access. It’s one of the more noble vocational pursuits to propagate honesty in a sea of redirection and rumor, and it’s something that can be enabled and aided by technology. Given my recent research and just general current events, I’m incredibly humbled that I get the opportunity to work on technology for crowdsourcing and spreading information, and so I wanted to address how we’re tackling the vulnerability of information providers with our tech at Ushahidi and our trainings at Internews.

journo-attack

In terms of self and source protection, Harlo and I have compiled some applications that can help with operational security for journalists, and this applies to citizen journos as well.

opsec-resources

 

In terms of source verification, CrisisNET has prepped a roadmap series of features to integrate the likes of TinEye, and twitter verification via TweetCred. To that end, the devs at CN wrote about this application of authenticity readings to the CN service vis-à-vis  the IDF/Gaza conflict  recently. We’re working to build more security into Ushahidi’s platform as well, and otherwise increase the availability of our technology through much needed translation efforts. Like most of the platforms we provide, we rely often on crowdsourcing and community participation to complete the arc of their utility, and we’re hoping our community will help make our products better.

Outside our own repertoire, there’s a beta product called Scraawl that also aims to provide streaming data about large scale graph and social media collections. There are further, plenty of ways to contribute to crowdsourced journalism projects: join Open Reporter, a platform for free and open news, or Open Street Map, a crowdsourced program for mapping the globe, or Project Fission, an open source project to manage reporters’ notes and stats. Opine and add-to where possible, open information and citizen journalism still source some of the most up-to-date coverage of crisis worldwide.

jihad

Meantime, I’ll close with a more positive piece, reblogged to oblivion, on Yemen, a link to some github to watch as we move more data viz into Ushahidi’s core, and request eagerly any blogs/sources to watch below:

Tagged , , ,

Parallax Effects: Modern Map Moving in Interactive News

yelp-map-mover-bubbleUntil a few months ago, Yelp featured a modal pop-up when users tried to click-drag the slippy map provided to situate search results according to a local topography. The prompt was “Hey Map Mover?…blahblah ” and I used to think it was another annoying modal, but now I think it’s a rather magical metaphor for the kind of awesome power involved in building visual representations of complex information, aka maps.

I think about this moreso now that I work for a company that builds maps, and think progressively about the map as a super-class of data visualization, under which all other data visualizations cascade. Maps are unique because they provide link to data collected and clouded online, and the physical topography of our global world; they bridge physical (real) and digital (surreal?) in the same attractive fabrication; they also provide a sense of trajectory, travel, and movement. I’ve written about this special quality of maps in some map-love blogs in the past, but its recent relevance seemed to warrant another post. It’s in this context of map-making and movement that I think about my job in Open News and open source software, and it’ll take a bit of a preamble, but this blog post is going to address all of those things.

Parallax Projects

I’ll start with a small anecdote. Recently, and independent of my fellowship, I had the opportunity to collaborate on parallax projection installation at the Museum of Holography on Governor’s Island. Like most of my side projects it was weird and largely web-based (see my radio show, or bio art projects for further reference) and was a collab project with another friend. My “real” work, which I allude to loosely and often on this blog involves development and data viz via a fellowhsip with Knight-Mozilla Open News, coding for for two organizations that build curriculum and technology tools for citizen journalism in East Africa. I’m passionate about all of these things, professional and peripheral projects. And as their linked by a shared trend toward constant iteration and movement, they’re not completely divorced from the parallax concept that drove my holography art project, if only because the moveable and mutable qualities shared between these things seem to theme them quite neatly and help explain why they are compelling, at least from my perspective.

more MerryTimes

I think we see parallax most often in the context of jazzy-layered web techniques, where scrolling interaction reveals movement via a multiplicity of tiled images. Likewise, ‘parallax’ is pretty important for all kinds of optical assessments, depth perception and distance inference; it’s a good metaphor for describing alternative perspectives on a distant trajectory endpoint, which by extension is a good metaphor for describing career paths and their pref-the-path-less-traveled discontents. I’m of the mind that things I do and enjoy are pretty awesome; though I respect that alternative perspectives exist. I thought I’d write a post to cover some recent work and play projects with parallax, a little description of my day-to-day, and the pretty awesome (IMHO) projects I have the privilege to contribute to as an Open News Fellow, in echo of my fellow fellows

It helps that ‘parallax’ a pretty popular buzzword these days on the web and in journalism. It crops up in NYTimes visuals, in Joni Mitchell-inspired xkcdsin topojson Bostockian demos; sometimes it can look like this, or otherwise like this. Check the links or inline images to see what I mean.

nytimes

My parallax projection project (say it, 5x fast) was not as intense or playful, but probably on the spectrum between those last two, it’s some solid hyperlocal-fun about fuzzy riverscapes and insular environments. The general objective of parallax visuals, ZUI interfaces, and anything that implies movement on a 2D screen is to immerse you in an environment or a static scene and draw you into the automagic of its motion. Constant motion, “agile” development, characterizes a lot of technology and journalism of late as well, and parallax has become a pretty popular and loosely applied concept in the realm of online interactives and visual data re:presentations.

more MerryTimes

Movement and the Making of Open Maps

http://andrevv.com/But back to how this relates to my current condition and career path plan. I think about parallax and movement a lot when I describe the agility and adeptness that my job requires, and the kind slurry of opportunities the Open News Fellowship provides.

On the daily, I juggle lots of things working for two software and data-driven journalism programs (Ushahidi and Internews-Kenya). My coding projects range from contributing to open source citizen journalism and crowdsourcing platforms like Ushahidi Version 3.0, or 30 year retrospective interactives studying the semantics of socio-political perspective on East African health, or providing visualizations of violence report data as part of election monitoring initiatives in Yemen and Nigeria, or plotting environmental sensor data from collection stations across Tanzania in a web-based viz series. As asides, I develop courses and curriculum for a non-profit that I help run in NYC called Girl Develop It and mentoring budding coders at the Academy for Software Engineering in New York. Tacking on the weekly radio show, and occasional art projects; I accumulate an impressively deep collection of random. Point being, I learn constantly, work assiduously and have the opportunity to meet at conferences and in collab code sessions with some of the most impressive, genuine and amazing humans in the tech/journalism space.

In my short (now 6 month-ish) tenure as a fellow, I’ve traveled to five continents, lived on two, attended over 10 conferences, spoken at 8 of them, built open source projects with collaborators at 5 of them. You can check my commit history on github or my fellowship repo where I’ve been tracking projects and speaking engagements, including my now rather abortive attempt to convert github issues into a todo list manager for my all of my talks and tasks. For now, I’ve been logging my ongoing projects, conference notes, and both present and potential work thereabouts, and so the version-controlled storage model provides pretty fair understanding of the constant change, abrupt but amazing shifts, prolific blogposting, and pretty persistent parallaxes of my fellowship program.

githug

And not to be too simultaneously overly-alliterative and Cindy-Lou-Who-twee with the parallax refs, but I think some of the most important developments of this experience haven’t been logged in a blog post or my tech-tonic shifts of my task documentation; some of the most important developments have been in a personal discipline I’ve mustered through these experiences, to manage ALL THE THINGS, prioritize my own workflows and organize my life to more maximal productivity. Part of the elegance of the parallax metaphor in this case is in it’s homonymous “pa-relax,” which can maybe best reflect in a portmanteau that some of the most sustainable lessons are those that allow you to partner the extreme opportunities and impressive responsibilities, with an ability to manage expectations methodically, and support your collaborators without stress or burnout.

MerryTimes, a parallax project at the Holocenter, NYC

This requires some flexibility, some skill, some patience with constant change, some humble understanding of your own personal limitations and a healthy refusal to be stunted by them; these are qualities I’ve developed more thoroughly in recent months, and recognized pretty profoundly in my fellow fellows. They’re qualities that everyone should seek independent of their embrace of a parallax life model or map moving philosophy. If you want to do great things, develop professionally, collaborate actively, progress personally, and contribute productively to a global or even niche-local community, these are qualities you’ll probably invest time in developing. Why not do so with some support along the way?

Progressive and Open (Source) News

To that end, you should probably apply for a fellowship like mine, though your experience won’t be identical. An Open News fellowship is simultaneously the most flattering, exhausting, and exhilarating honor, and jobs after it will be hard to reconcile with the epicness of their precedent. My goals prior to the fellowship were pretty basic: code more in the open, Holocenter wall textlearn more about design patterns and backend development, develop sustainable architectural habits instead of one-off project drill behaviors. The scope of my current learning and side-project -> tech stack spread is pretty impressively expansive compared to my initial projections.Hologram Glasses at the Holocenter

testing the parallax art app in situSome of the more tacit benefits are nearly impossible to articulate without being gushy. It’s the stranger famery you’ll experience in the news community that clashes with your impulse to imposter syndrome; the kind where you’ll get requests to collaborate on projects from strangers instead of just your friends. Pre-fellowship, I never really had comments on my Github projects and my public code persona was pretty weak; 5 months in, I get regular email about blog posts I’ve written and repos I’ve open-sourced. I speak at conferences where people preface their questions to me with “I read/looked at your…”, instead of just “hey so I disagree and can prove I’m smart by asking an un-question.” (We’ve all witnessed these people; I witness them less now).

my beta-BRCK, ready to test!As an added perk, I get to work with people who develop technology solutions for under-tech-privileged parts of the world, places where access to the internet might require cell network fail-over solutions like BRCK, or where tracking citizen opinion via social media streams aggregated and API-ified in CrisisNET is so epically important because these opinions are routinely throttled by oppressive or inegalitarian agendas. I beta-test and help build the budding work that will change how we process open information and how we crowdsource commentary. I work with people who train journalists on the ground in nations where objective journalism can be unwelcome, and the practice of building a data journalism program is as-yet nascent or continuously undercover. I’ve had the opportunity to fuse collaborations with the brilliant trainers and journalists at Internews and the smart and savvy technologists at Ushahidi; and in this capacity, I work with and support people who develop new ways to provide critical perspective on sustainable sociology and global health programs, on geopolitics in the absence of privacy and in the face of human rights violations.

Ushahidi V3: Soon with more maps + graphs!

I think about this more and more in the context of how journalists have adapted to suit these demands, becoming multi-tasking reporters/technologists/domain-specialists/data-“scientists”/activists. A recent BrainPickings article about Achebe’s approach to writing as a vocation seemed to fit here too, where “writer” adopts all of these newfound technical roles while maintaining the same ethical mandates, explaining what Maria Popova so awesomely cited as the “redemptive power of fiction and Chinua Achebe’s perspective on optimism in lieu of ‘clickbait’ journalism”:

[My idealism is] still alive and well because without it the business of the writer would be meaningless. I don’t think the world needs to be told stories of despair; there is enough despair as it is without anyone adding to it. If we have any role at all, I think it’s the role of optimism, not blind or stupid optimism, but the kind which is meaningful, one that is rather close to that notion of the world which is not perfect, but which can be improved. In other words, we don’t just sit and hope that things will work out; we have a role to play to make that come about. That seems to me to be the reason for the existence of the writer.

My work is fulfilling, optimistic, open, and ever-(map)moving. Yours could/should/would be too.

MerryTimes, 2014

Don’t wait
August 16 is the deadline date!

(; I drop rhymes like this errryday ;)

Read all about it here | Apply here

Ping me for any questions here: aurelia[at]ushahidi.com

NYPL Labs also has a map-makers residency for interactive artists, if that’s more your bag

Tagged , , ,

There Must Be A Pony Somewhere: Digging in Data to Find a Story

CartoonicornQuote investigator wrote a cute quip about the origins of this blog’s title quote (“…there must be a pony somewhere…”), and lately, it has me thinking about a job I share with many techy-journalists: digging through data (evidence) for a story (pony). I’ve commented on that a bit exhaustively in this blog, but the metaphor carries through to building a data journalism team, composed of a ragtag herd of unicorns, racehorses, and predominantly, ponies. Online Journalism Blog did a short piece about the taxonomy of journo-developers too, bulleting a few typical types (racehorses, unicorns, mules),  to which I’d like to add ponies before diving a little deeper into what this means in terms of characterizing a professional population by its equine analog.

At this week’s MIT Civic Media Conference, Joi Ito kicked off an introductory talk with a nod to his coder fellow, a “unicorn” journalism-coder-analyst that had just joined the team, so the metaphor has stuck with some steady citation and I think it’s worth discussing here. In the next few sections, I’ll cover a few adventures in geo-journalism, talks and projects I’ve done around mapping in the past months. Moreover, this will be a blog about our equine habits and heros in data journalism, and some musings on what media hackery earns in terms of recognition and reward.

Dev-Journo Taxonomies

zebracornThere’s an understandable spectrum of personality types and professional competencies in Data Journalism. There are the fantastic anomalies: unicorns; the hardy worker hybrids: mules; the strange and rare portmanteaux whose skills define along a folksonomic schema: looking at you zorse, zebroids, donkras. I gave a talk on Data Journalism a few months ago (check vimeo below), and the thesis of my presentation echoed the essentially hybrid aspects of the job.

Those born under the sign of the Horse are a flexible group of people. They tend to be stubborn when it comes their ideas, but they are also incredibly patient when it comes to hearing out what other people have to say. They favor straight-forward conversation, but avoid trouble where possible; a paradoxical combo, but one that makes the horse persistently fascinating as a sub-population in the animal kingdom.

Data Journalism in DR

So in the data space, why fixate on ponies as representative of some substantial sample population in the greater software engineering venn? Because ponies are slightly different than horses; capable of the same intelligence and empathy but perpetually twee-er and often assumed to be less mature. Some of the brilliance I’ve witness from millenials in the data journalism space has made me think that another branch from the taxonomic tree should recognize those whose aptitude is impressive in code but whose journalism background, and experience in general perhaps seems premature.

Pony Projects

muybridge-2When social media steps down from the free speech party, and while governments and institutions of modern social exchange continue to use networks as a way of monitoring and managing society, it’s often the critics and the activists who have to pick up the slack to produce objective publications and in this space the post-modern (and often, outsider/premature) workhorses of the data journalism space have something to contribute.

As a class, proto-journalists and data mungershave developed some tools to analyze trends and provide objective and dissected-unicornuncensored criticism of the information they represent. Zeynep Tufekci’s talk at this year’s MIT Civic Conference on citizen investigative journalism in Turkey gave a nod to the use of social media (and twitter feeds in particular) as infrastructure for collecting public opinion and fact-checking specious claims. Many tools for crowdsourcing, Ushahidi included, can be deployed to provide for citizen journos-ponies, smaller breeds of self-taught but domain-proficient reporters, with tools for reporting. And while much of this citizen-driven practice is perhaps under-promoted in the contemporary news space, some of the most renegade journalism efforts are sustained by citizens running depolarization operations on social media platforms in their home countries, as Zeynep’s talk suggested.

Pony Hierarchies

 

Part of the persistent argument in discussions that blend net neutrality, privacy and surveillance censorship revolves around how important crowdsourced and social content has become for developing honest and unbiased alternative reporting models globally. Though not to be confused with incident data directly, social media reports like CrisisNET’s Syrian Youtube Map and Conflict Map’s tweet and social media tracking plan provide these kind of windows into the world of social streaming to study crises. In analysing, contributing, and disecting social media content, pony-journalism has become a more dominant approach to assessing conflict and geo-journalism at a global scale.

Muybridge Motion Studies

In fact, arguments around how to classify the oft-hyphenated and obscure titles applied to data-journalists are more about the hybridity of their job descriptions and the range of skills they deploy than about the elegance of the metaphor. As an equine-hybrid class, we’re often trying to find new ways of developing and pushing content, a nod to the aggressivness and tirelessness of the horse behavioral type. But part of that race, maybe the most important part, is about designing content and news to appeal to people, to visualize data in new and yet intuitive ways. Our objective is to find ways to relate to populations, and in a sea of bar charts and statistical models, sometimes maps are the more affective way of relating complex digital data to a simple physical topography. That’s where the map making (mentioned above) comes in.

fancyTwo of the most relatable and persistently referenced data types in post-modern visualization are geo-data and time-series. Why? Because we relate to them, we can consider our perpective relative to time and space; they have become our touchstones for syncing digital and physical worlds. Overwhelmingly, the projects at this year’s Civic Media Conference demo sessions fell into some kind of mapping context, and I think that trend is telling for the direction of visualization schema and citizen journalism: What We Watch, a map of youtube trends; Terra Incognita, a Chrome extension for mapping exploration; Media Cloud, a collection of tools for monitoring and mapping media globally; or Cliff, a project to automate media geo-parsing, being a few among many featured projects. Tools like We Feel and CrisisNET are aimed at facilitating this kind of study, enabling study of social media and reporting strategies. In each case, it will be interesting to watch how they compete in the investigative reporting space; the race seems primed to recognize their utility.

Pony Prizes

BookAnimalsTo address another interesting aspect of the data-journo ecosystem, I’ll now pivot to another curious theme in the MIT Civic Conference and others like it: the concept of work- “family.” In keeping with the metaphor of this post, and I would argue that family in the case of a company or sponsor, is more analogous to genus hierarchies than to social kinship models. People who share a company share a type and a goal, they’re a team but one built on affinity, not consanguinity.

This is a family:

IRL family reference

This is a team:

Company-Family

Company-Family

A company/funder/sponsor/laboratory/media-outlet/workplace is a herd of ponies. As individual members, we are unique in our methods and backgrounds and generally attracted to the same trajectory, but probably more powerful in that dispassionate diversity which a team or herd-mentality affords, less complicated by emotional entanglements internally and therefore more competent at empathy externally (that is, with our users/subjects/sources for stories). In a recent HBR article, “Your Company is Not Your Family,” the author uses the analogy of sports teams and the mentions of the spurs made me think that the pony metaphor might be as ridiculously apt.

The Spurs stand out for the stability and longevity of their player relationships, yet even their current 13-man roster only includes one player from their first championship in 1999: power forward Tim Duncan.

The PrinciplesTo consider your company analogous to your family, is to cripple it by a lack of adventure. Families, while wonderful, are a default, they usher you to growth, but if all goes well, you flourish on your own. You want to build a company of people who are flourishing, and will continue to do so under guidance and not parentage.

Joi Ito concluded the MIT Civic Conf with a series of “guiding principles” at the media lab, and those statements reinforce all-the-more why a lab/company isn’t a family. A team can be built on shared principles, but they’re not the same as those on which a family is founded.

Follow your unicornnon-believerYour family pushes you, educates you, and prefers (often) your safety over risk taking, whereas your work, and your class (genus/type/subgroups) often push you to independent and outlier achievements unsanctioned by precedent and rarely “safe” in practice. A total aside in this blogpost, to be sure, but I think often data journalism professionals (and by extension, other political/social-professionals who put position before the public they serve) seems to allow confused allegiance to cleave them from simple human and social empathies.

This is a point I treated in a recent interview with Danish news about the relationship between developers and journalists. Nothing revolutionary, but at the time I compared the ideal scenario to one of mutual respect in difference, and not to a familial metaphor. My collaborators aren’t my siblings, they’re my colleagues, and the relationship is pretty different in my mind.

We sometimes risk an allegiance to an editor or organization over an allegiance to the public, and it’s important to remember that the protection and privacy of your subjects and sources is just as precious as that of your employer-parents, regardless of who is paying our salaries. Too often, I’ve seen people at conferences too proprietarily motivated to share ideas, too proud to admit that many share the same ones and have started similar projects. There was a lot of overlap at this year’s Knight News Challenge award announcement, and I think it’s fair to ask overlapping orgs to collaborate and share their plans and programs of research as the year progresses, though I doubt they’ll be held to this. Sometimes, considering your company like your family can confuse your objective to do good in the world and supplant it with one to do good for your own.cartoonicorn1

This brings up another aspect of social good work, and journalism worth mentioning here. Often, the competition in the data journalism space is built on a capitolistic motivation to secure funding and support and resist the superior publication of another outfit that prematurely scoops your content. In this fear, we privilege our company over our vocation, which is to spread solid news, to share it with the world. There’s no shortage of conflict and controversy worth commenting on, so the competition seems sad and contrived especially in the social good and open source space. But recently, I’ve been reading economic coverage of the pay-gap issue and have come to appreciate that this competition has deep roots, founded in our cultural resistance to recognizing social-good as grant-worthy.

unicorn shower

some related items found on the Pinterest “unicorn” keyword search

 

The most prize-worthy ponies deserve reward, and I think it’s interesting to consider how we approach compensation when the goal of your work is social good. The resounding answer seems to be: we don’t.

Econ-Theorist David Graeber’s recent interview on the trends in our financial sector indicates that we rarely value work performed with altruistic motives, and that we waste most of our workforce on “bullshit jobs.” While our intentions might be genuine, study of our current workforce specialization schema indicates that we dole out few directly productive (as in “product-building”) positions, and most work is “administrative” or “managerial”: “…[l]ots of people [in Graeber's interview pool] said their basic function was to create tasks for other people.” One quote that struck me as particularly insightful:

Geoff Shullenberger recently that pointed out that in many companies, there’s now an assumption that if there’s work that anyone might want to do for any reason other than the money, any work that is seen as having intrinsic merit in itself, they assume they shouldn’t have to pay for it… ~David Graeber

You can read more about his provocative, and well-argued perspective, here, and while he applies his study to translations jobs, I think the scope can widen to anyone doing fulfilling, socially-conscience, and context-driven journalism, globally; we’re all in the information translation/transformation/communication business at root.

You know, you’re describing what’s happened to journalism. Because people want to do it, it now pays very little. Same with college teaching. ~ Thomas Frank

Upshot: not compensating people doing good, critical, and socially beneficial things in the world is crippling our perspective on geopolitics and progress.

Problems with Ponies Abroad

Other than economic obstacles to pursuing social good, there’s other hiccups to the hierarchies of investigative journalism that relate to how we privilege unicorns over the content they cover, and here we return to our discussion of mapping. When I was at a hackathon last month in Aarhus, Denmark, my team won the Guardian API award at the event not for building something incredibly revolutionary, but something quick that simplified news content into a digest for mobile journos.

ecard-horse

Our app was called GeoNewsies, and its objective was to allow travelers to search by country and pull down a digest of the news in that nation prior to, or during travel. A two-paneled webpage and android app, it pulled in the top 10 articles from the Guardian relative to a particular place (panel left), next to the top trending tweet topics in that place (panel right); a bit like thenews.im or other rss aggregate sites.

geonewsies-web

The interface was unstellar, simple, and arguably flattened the geo-political happenings in a place to a top 10 trends list, but our objective illustrated something tragic and important about how we process news media today, and maybe it’s not what you would expect. Our point wasn’t that people only can afford to read short blurbs and dramatic reductions of the richness available in pre-travel research, but moreso: often, travelers fail to self-educate about the context they are about to enter, and this unfortunately extends to even traveling journalists working investigative beats abroad.

ecard-unicorn

Sometimes, the best witness to activity in a particular place is someone on the ground an local; this is why so much social media analysis and source relations with citizen journalists remain important to our global understanding of news. Displacing a data journo-“unicorn” to code in a foreign environment is rarely as productive as sourcing information and accounts from the local population, and then enlisting the unicorns or racehorses to usher an idea to production; or better, training the local ponies and mules to race.

http://www.buzzfeed.com/donnad/stop-what-you-are-doing-and-look-at-ponies-in-swea?utm_campaign=socialflow&utm_source=twitter&utm_medium=buzzfeed

Scotland Tourism’s Sweater-Pony Campaign

Burak Arikan’s MonoVacation tourism visualizations speak to this touristic approach to documentation of place that has become our practice in journalism. Arikan built a projected mashup of the tourism video/commercials of many nation, exploring typical symbols and their geo-contextual meanings relative to the nation of video production. Horses were a trend, repeatedly used in travel commercials to express freedom and tourist wimsy, perhaps. Abstracted a bit further from the original project focus, and deftones - because obviyou might consider the horse comparison to data journalism as a sometimes apt description of investigative practice: short sprint production and reporting with often unfortunately abbreviated context: a tourists’ view of geo-politics. Often a foreign media outlet’s assessment of the on-the-ground occurrence in one place lacks the depth of historical and hyperlocal understanding that social media reporting/analysis can provide if controlled, curated, and harnessed to meaningful ends. Oualwaysr attention span for international news is something that perhaps can’t be corrected but our approach to economizing a broader range of opinion and local perspective is something that might be best achieved with social analysis and local data journalism training.

As someone who came rather late to code; I’m pretty comfortable advocating the premise that code can be trained, and not limited to the hierarchies of mythical creatures. I’d argue that researching for a story involves a healthy amount of logic that is more intuition and contextual/location knowledge than technical skill. Compelling news applications about a particular time and space are ones that root in a thorough knowledge of the geo-politics of a place, and often those come through most clearly from content generated by local mules, rather than unicorns.

Post-HorseRace: Project Persistance

equus-evolutionIt’s safe to say, however, that team assembly and the logic of our production pipeline aren’t the only concerns in developing sustainable news applications. With news apps, we deal in a particularly friable media; one whose impact often limits to the extent that it’s API/library/dependency components have yet to deprecate. When we think about endurance and the persistence of applications, we sometimes think about the ephemerality of our work.

What happens when the horserace is over; how will we remember our efforts?

This worry is not new of course, and its one that’s been persistently suffered by media producers and providers globally. Born digital projects are so vulnerable to almost immediate atrophy, and while you may make history with a web-based piece; the probability of it outlasting  even newsprint articles from 30 years ago is pretty pathetically weak.

We’re tackling that next month (July 23rd) at the 2014 Digital Preservation Conference in DC, if your’e interested, so check it out. Our objective in presenting is both to survey the state of media production today and discuss preservation options, but also acknowledge some technological trends we should avoid. Contemporary product development is replete with light-bulb conspiracies of ‘planned obsolescence’ and at the opposite spectral pole, stories of technology built for eternity. Somewhere in the middle, there’s a place for news apps in our geo-political history; a few pony programmers might just figure-out how. :)

Finish-Line

ilovethisTo sum up this (rather-too-longform) piece about pony personalities in the geo-newsroom, I’d say that a lot of our professional expectations as journalists and developers presume a few narrow ideas: firstly, that a simple taxonomy can define competence in global news coverage, secondly that companies can operate like parents, and thirdly that the integrity and sustainability of your work are secondary considerations to the general scheme and scope of a path defined by paternity.

I’ll close with a link to my MIT Civic Media Ignite slides (presentation, references); it’s a talk about teleportation and mapping, but no less fantastical than the expectations of data journos globally (that we tell the future, that we perform our pony tricks on demand, that we manage to t[rans/ele]port). An area of growing interest in the data journo world is how we manage to create compelling narratives about remote happenings, and often these are through our modern tools of teleportation (things like Ushahidi’s BRCK or OpenNews’ Keyblur for deploying networks without Internet, or applications like Crowdmap, CrisisNET, and Media Cloud Focus, helping us to understand global coverage and crowdsourcing context from operatives on the ground. These applications are among the suite of devices at our current disposal for feats of science fiction fantasy, bringing our ambitions of teleporting and unicorn reporting all the more close to our realities of remote monitoring and pony-journo practice.

 

Tagged , ,

Radio Gaga: Spectrograms + Cover Songs

eyeoThanks for coming to my site!

I gave a quick Ignite at EyEO this week.

Some wiser person once told me to never give a talk that should have been a blog post. So to solve for that I’m doing both; less is more just less. There’s always so much to say in presentations, regardless of the constraints (5min time limit, frozen audience, IT issues), and some of those things can best be said when you’ve had the opportunity to reset sanity off-stage. This will be a quick post about what I did for the talk, how I did it, and why. I put the links at the top, so you don’t have to read everything.

Screen Shot 2014-06-11 at 2.05.21 AM

Read on below!

Continue reading

Tagged ,

538: Errors, Plotting Crises, and the Protocol of Re:Processing Data

There’s probably an HTTP error code for every situation; for this post, 538 seems to well-suit. It’s a Windows error that returns a dialog about ABIOS (Basic I/OSubsystems, indicating invalid entries and corrupted drivers. Despite their obscurity to most of us, these are actually common and analogous issues in developing data projects for journalism…corrupted, dated, or invalid info being problematic in both cases. This is a post about one of those cases.

ABIOS ERROR CODE

If you’ve been following journalistic tracking of the Nigeria kidnappings, then you might have come across 538, a collective of hackers and journalists who has been reporting on the topic and recently posted this set of maps using GDELT (Global Database of Events Language and Tone) data. This garnered a series of pretty solid rebuttals about integrity of their assertions; see @charlie_simpson’s Storify feed and Daniel Solomon on Source. The problem with the piece in question (to summarize the previous links), is that it provides time-series and mapped analysis of kidnapping in Nigeria but skews representation of the actual data plotted.

 

 

 

 

 

 

 

 

 

 

 

 

As someone who works with journo orgs, crowdsourced crisis-mapping projects, data, and Africa I thought I’d comment on some of the fallibilities briefly. The particular fumbles I see in the 538 representation of kidnapping incidents in Nigeria can be bundled under three issues that are persistently problematic in all data journalism projects.

ISSUE 1: REPRESENTATIONAL INTEGRITY

A lot of issues with data mapping/graphing projects boil down to human representational error: what is your map actually showing and what are you saying it’s showing? In this case, the equivalence of GDELT media data and actual incident data is a superfail, but not only in the (mis)representation of the source used. The failure to buttress that representation with clear disclaimers and other data is also unfortunate, worth commenting on here. Quotes below taken from the 538 article in question.

Official kidnapping statistics for Nigeria aren’t available, and our numbers do provide a good relative picture; we can see where kidnappings in Nigeria are most prevalent.

This points to data paucity, which is fair, definitely a speedbump, but not entirely excusable. We’ve been spoiled perhaps by the assumption that everything should have a .csv download or an API endpoint, or that you can get all of the things from one aggregation feed, but some more context here would help.

The link in this quote, for example, should be bracketed in context, linking to a 404 (“aren’t available”) like this is unhelpful when you don’t know the query that led to it:40Fail.superlame.com

 

What about showing why/how your query was unsatisfactory? If you search in prognoz (the Nigerian Statistical Open Data Portal used the 538’s author to search) you do find data under “Public Order and Safety” as a data category, indicators (search terms) like “kidnapping” result in graphs from 2006 + .

Likewise, if a trend in one data set is notable, particularly a geographic density of “events” on the map, it’s worth looking at other data to supplement your assumptions.

One possible explanation is the region’s oil wealth, otherwise known as the curse of the black gold. The United Nations news service has also highlighted how oil extraction in the south of Nigeria has been accompanied by violence and criminality.”

If a relationship to oil by region is of interest, Prognoz has data for that (Macro-Economic Data > Petroleum), or maybe there’s another relationship to geography worth exploring: topography, environmental influences. Perhaps a comparative analysis with other mapping projects devoted to those data, like Oil Spill Monitor – Nigeria or flood tracking and standing water in the regions where 538 notes a density of kidnappings would be of comparative interest. Are there other geographic factors that might affect crises worth exploring?

There’s a value in layering data sets and comparisons across mainstream and social media, and the real value of journalism’s take on these data is the comparative perspective it can provide, recognizing the weaknesses between data sets and using them to crosscheck each other rather than only “normalizing” to control for error in one set. 

This is a somewhat crude calculation. We’re counting all geolocated kidnappings in the GDELT database since 1982 and dividing that by each state’s current population.

So, does that mean that the current population in a region was the denominator for that division across all decades (because at the time of this post, the population link provided in the post doesn’t load)? Where is the data? how can people access it, can I get a tooltip with counts and calcs in the timeseries (pretty sure cartodb supports this; I mean, really, man.)?

ISSUE 2: DATA BULLETPROOFING

This is the predominant criticism in both rebuttals, the refrain of all journo projects being a pretty neat alliterative philosophy: check, compare, contextualize.

Validate Your Data

As this has been well-covered by the other critics and is a pretty well-documented challenge in journalism (see: “verification by replication,” scientific method-style), I won’t belabor it here. Qualified outfits have written impressive how-tos (like this awesome one from ProPublica) though the process for bullet-proofing each piece is usually custom. There are also papers and projects like the Data Verification Handbook, and applications like Twittcred and Storyful aimed at affirming social media.

Early in this bullet-proofing process, it’s also helpful to take a look at comparative projects and use them to illustrate why your analysis is distinct, and how it contributes to a gap. Nigeria Security Tracker also has mapped violence and fatalities in a time series; Nigeria Watch provides a database of violence trends as well, and there are other authoritative and georeferenceable event data with downloadable datasets worth querying against to better verify GDELT.

ISSUE 3: SOCIAL OR SECONDARY MEDIA AS SOURCE

Lastly, and predictably, there are always hiccups when plotting social and secondary media accounts as events.

what GDELT *will* tell you

Analytics on postings and general media circulation can be valuable for viewing the conversation around a topic online, but they can also be speciously spun to represent the density of actual crises or activity in an area. Counting the tweets related to #nigeria isn’t entirely useful for modeling a threat without filters or ways to validate those postings. Even GDELT, in its ambitious programto provide the global research community with its first open global multi-decade quantitative database of human society” is still researching how to best verify social data.

Let’s look at a more general example mapping data. GDELT represents media activity around topics, like how google trends represents search activity on topics, but both can be confused with representing incidents. In the later case, examples of secondary source and interpretive fumble abound. 

Take Flu Trends:

 

or this Google Trends graph of a few JS libs one (note the rise of Angular JS in recent times):

Angular vs. all other JS

What these graphs illustrate is not an actual density of flu incidents or a spike in public interest in Angular JS but rather the number of searches related to incidents, and perhaps public confusion about Angular JS. People who have the flu might also go straight to the doctor and not google it; people who understand and appreciate Angular are perhaps unlikely to google for Stack Overflow. Media discussion or focus on a topic does not always/often equate with actual activity, though the two are sometimes conflated.

Just as there’s a tendency to consider a social media campaign as solely-sufficient involvement in a crisis situation, there’s a tendency to tap a feed aggregation or media API as an authoritative representation of actual events. The distinction between social and mainstream media fuzzes when mainstream relies on social or secondary media as data, a problem in the 538 case, as they provide analysis of an aggregation feed of secondary media accounts of events.

Often, social media is incredibly powerful for plotting the general conversation about a topic (I’m looking at you, Westgate twitter tracking). Some of the most positive reactions to this crisis have been piloted by social media (#BringBackOurGirls), whose impact can be limited practically, but potentially epic as an indictment of the the government and mainstream media are doing comparatively. There’s little that’s less shameful in our digital world then having your government and formal press upstaged by hipster hashtag advocacy. That’s not to say, certainly, that these campaigns aren’t subject to their own epic blunders of failed verification (see: #yikes).

But beyond press campaigns and historical analyses of population/kidnapping trends, projects that pull in crowdsourced data are pretty impressively valuable for soliciting first-person information and sparking citizen-driven initiatives; Reuters’ blog just covered a bunch of them as relevant to the plight of Nigeria’s current victims. Ushahidi, for example, uses crowdsourced first-person reports that have been subcategorized and mapped by the admins of each instances’ deploy. It’s not a perfect representation of conflict, and it certainly has its limitations, but it is a distributed 1st-person reporting mechanism that can track violence relative to a geographic location depending on how the instance is customized. Secondary processors of this information can add a layer of interpretive error that weakens the integrity of the sources, if by only failing to admit their fallibilities. There are several Ushahidi projects that track violence in Nigeria, with their own foci and categorization schema (distinguishing between “trusted”/”verified” reports and public feeds). Like Niger Delta Watch, or Extrajudicial Killings – Nigeria, or Stop the Bribes, all of which provide first person accounts of violence as mapped to regions in Nigeria.

No one is be perfect all of the time, or capable of pleasing all the people, certainly. GDELT is an imperfect source of most things beyond tracking media reaction, so it fails in this effort to echo its output back as event data (see Source). However, media reaction is still interesting for other analyses, hence the media reaction to these maps; the integrity of a news organization and its output of (even aggregated) content is still worth indexing.

EOD, the ethics of data journalism and best practices haven’t been adequately codified for these kinds of stories. At last year’s Highway Africa conference, Peter Horrock (BBC) talked about the best indices of quality media covering Africa being somewhere at the intersection of how an organization covers domestic events and how it covers its mistakes (see his full talk here). In this latter case, media reaction is important, if for a different reason. We’ll see how 538 reacts, and maybe learn something about how to manage future code-fumbles. I’m looking forward to more verification protocols: representational integrity, data bulletproofing, and secondary sourc-ery ;)  </ERROR>

* Thanks to J. Morgan. E. Constantaras, and  J. Rotich for contributing data, time, and thoughts to this post

Tagged , ,

That’s What She sed: !awk Lessons From Fun[ctional] Programming

Somewhere at the intersection of unexpected genius, linguistic mastery, and femininity there’s a trope of compelling film/fiction that goes something like this: a character (ideally a woman or weakling) speaks a language that no one expects and suddenly reveals a competency or comprehension that strengthens his or her position, provides for some comedy, or drops a beat of provocative timing. This kind of surprising exolingual + monolingual situation is common and interesting. I’m thinking Daenaerys Tarygarden speaking Valerian, or when Nancy Travis speaks Russian to her cat-callers in So I Married an Axe-Murderer, or that scene in the Goonies when Corey Feldman, a child, gives the maid surprising instructions in Spanish, or those times on the subway when I can tell who the françaises next to me are gossiping about and giggle to myself at the semantic secrets I’m privy to by virtue of closet bilingualism. It’s a common and compelling scene, not one wholly relegated to spoken tongues; it has its echoes in computational languages too.

Unexpected fluency in a programming language is fascinating. There’s still an interesting amount of surprise that accompanies any woman speaking intelligently at a tech conference, or a child-ish programming prodigy who sells his company at 18 and enjoys wild and precocious success. With that in mind, I decided to explore some languages recently that I had little experience with, if only to investigate their utility, and build up some surprising and cinematic techcred of my own.

Ontology Web Language? (http://www.w3.org/2001/sw/wiki/OWL)

Informing this, a recent and short tumble into the land of Game of Thrones led me through the wikipedian labyrinth to LCS, this linguistic non-prof that constructs languages (conlangs), composed of member constructors (conlangers) and the responsible creators of languages like Klingon in Star Trek and Dothraki in Game of Thrones. My tangent into a trope sparked some curiosity about how we define computer languages and how we use them thereafter, and the authority of the inventors of these languages.ll-sarcasmantics

Like other languages, computational tongues are often indexed by stereotypes, but unlike spoken conlangs which have evolved to express a multiplicity of (in)translatable nuances, CSlangs often are more objectively restricted by a purpose, not developed to express all of the things but rather to accomplish a task. Valarian is “the only language for poetry,” while Dothraki is harsh and gutteral like its speaker population; SQL is a “special-purpose” query language, Objective-C is a “general purpose” object-oriented language, Visual Basic is the “most-WTF-y” language; but even in these stereotypical distinctions, coders contend about what these adjectives might mean, and who is best suited to speak these languages in such-or-such situation. And personalities presumptions align to these -types as well: women, being generally lovely and fluffy, are unlikely to speak a brutal and ugly  bash shell scripts….they should be front-end programmers because pretty, and easy. :(

And despite this, I’ve been investing a bit of time in the prelims of every data project, somewhere between scoring a raw pile of data and shaping it up for a visualization, always accomplished via some language/library. New projects and experiments always make me wonder if there’s a better library, plugin, resource or language to articulate my objectives and otherwise get me the results I’m after, which in this case involve a bit of OCR and semantic analysis, batch processing and cleaning and file pruning where language all-around is pretty important. For this round of tech adventures, I settled on SED, but I’m sure the operations I’ll be performing in this post could be fairly accomplished by other languages. What he sed.Further notes on my actual adventure can be found here, but as a quick suite of examples, say you have a batch of files whose extensions you need to change:

blog-origLS

You can do this with text utilities:

blog-textutil

This converts all .docx files to .txt in a given directory (ignore the bogus .pdf dud).

blog-textconvert

Then, say you need to restructure file names in a directory so that you can sort them, as I wanted to by date, but your current file format is something like this:

23NY080214.txt Or ##-NY-DDMMYY.txt

You can reorder characters in a set of files by running a sed script like this:

blog-sedreorder

This tells Terminal to break up the file name by “.” to represent characters and then re-order those parenthetical entities according to the numerical set order at the end of the line (4\3\2\1) where 4=YY, 3=MM, 2=DD, 1=23NY. It makes that reorder actionable for each (*) .txt file in the directory.

blog-rename text

None of these applications is really what sed was “made for,” but I found them pretty satisfactory implementations of the language for my immediate need. Taken together, all this got me thinking about linguistic development and about the “meta”-languages of programmatic thinking, the classes and cases of computational articulation that lead us toward fluency in one or more languages, preference, and eventual specialty in the operations most suited to that lexicon.

newLangsWhile living on a continent with ~3,000+ spoken languages, pidgins, and regional dialects, I also started thinking about how the diversity of computer languages compares to other paroles of parlance, and how our systems for organizing and inventing new tongues might best map to eachother for optimal productivity. There are rough guides for this kind of crosswalkexpected hierarchies, rankings, paradigm comparisons, and schemes of which languages are appropriate for the most hardcore hackers (see also, the “Real Programmer” fallacy).

But to redirect the conversation to a more critical and less-subjective breakdown, it seems appropriate to consider the semantics of not just the language itself but also its classification schemas in trying to assess their flexibility and purpose. One of the beautiful things about objectively breaking down languages by purpose, is that they can be ranked according to their flexibility and utility, their merits, rather than subjective judgements about their syntax. As with most anything in code, bash, or whatever scripting, part of the learning process is absorbing typical commands and the rest is playing with how to appropriately pair them for more complex operations (roughly: what commends are possible and how to link them). Snooping through Stack Overflow can usually get you pretty far on the first one, the second comes later, when repeated compartmentalized operations become exhaustive and your frustration has driven you to the point of investment in some serious study or thought on how to most efficiently arrive at your goal.

comp_linguistics

languagesFor this project, I selected sed because I’d read about its utility for my purposes. I’ve got several years worth of newspaper and journal data to convert from various file formats to one, and then rename in a batch before diving into the actual contents and cleaning and reformatting. Sed seemed appropriate for this, I could probably do it in Python or bash or JS or somesuch and maybe there’s someone who’s already build an online GUI that automates all this…but I was looking for something that worked and something new to learn, a new dialect to surprise myself with. While I felt stupidly proud when surprising others with this workflow and earning the ‘hacker’ merit badge du jour at work, I didn’t choose it to be cool, I chose it because it fit my needs. I chose it because sed is simpler than awk an perl, syntactically and performatively, but it provides a variety of text processing and regex support operations, and suits most things I would need in combination with other commands. I’m still at the ‘hello world’ stage with some of the magic of stream editors, but sed had some pun promise for the title of this post so I thought I’d go with that and see how far I could get with the operations that I wanted to perform.

And this is where I started thinking, perhaps there are other language paradigms to adapt for this purpose. Taking tips from symbollic and declarative languages might be useful, if only conceptually. I’d like to type in my desired output and allow the language to fumble through the mechanics of its implementation. When in SQL and I’m select from where’ing, I’d like to sed-ify that operation for data cleaning. Select *.csv from _ directory where _[date].csv. In researching and polling friends about addtional “sql-ish” (pronounces “squish” please) languages, I came across a few interesting features that I have yet to test in practice but seem like pretty cool operations to incorporate in a meta-sed lang.

In the past, and via wikipedia, I’ve heard  “declarative” applied to XSLT. Your blocks ll-intentof code are statements, declared like: “when you get to {this} w/ property {that}, do {these things}.” You can declare them in any order and they will run in the appropriate sequence.  However, is XSLT “declarative” according to all definitions? Diving further down the language research rabbit-hole had me questioning more of what “declarative” means in this context. Despite the overwhelming arguments you can get yourself into when defending the merits of one computer language over another, the terminology used to rll-morphefer to different programmatic concepts and classification schemas can be vague, misleading and largely unhelpful if you approach them as a foreigner, with other linguistic fluencies influencing your translations. The term “declarative language” for example can reference “non-procedural”, but that is also valid for the other language styles. The author in this article linked above uses “where you declare…” to define his term of “declarative language.” With XSLT, you write blocks of procedural code, called in reaction to something in the source doc, otherwise unlinked to the calling procedure (“where you declare…”).

If you think of lots of front-end and web prog languages, they pretty much fall into this category: small blocks of code linked to a user interaction, operation (onClick listen –> then run {this}). The author features a bunch of interesting language paradigms like concatenated languages, but there are other, now (perhaps) obsolete meta-languages that also address these concepts with more flourish and in many cases the same hiccupy classification semantics that can obscure their utility. Like what about languages made to describe algorithms, APL-ish tongues with general and placeholder operators, “compression functions” to apply operators pairwise to members of a vector, right to left programming execution sequencing. Or what about REXX, a shell scripting language using juxtaposition and ‘|’ interchangeably for concatenation, using blanks as operators. Even the semantics of concatenation have been through debates about the appropriateness of the term to “co-chain” vs. just catenate (“chain”).

Both conlangs seem to require quite a bit of syntactical adjustment but have features I’ve never seem echoed in other languages. And still, the point is, no one remembers these syntactical idiosyncrasies, languages are remembered for what operations they perform and how well. Our memories are operation-orientated, perhaps not-solely focused on syntax. Are these lexicons appropriate for high poetry, are they guttural and direct; what do they evoke, how do they surprise?

Plus, I’m wondering if I even understand how to appropriately use and manipulate a language when I’m not sure how to best describe it. Taking a page from my spoken fluencies, those languages that I know best and feel most comfortable using in practice are always those whose grammar and constructs I can explain and justify with greatest ease. There’s little mastery in the unwritten blundering I do in Swahili or Creole, though I’ve spent serious time in places where they were spoken; English and French, the product of formal study and informal fumbles, I totally own like whoa.

lang

In programming there’s a declarative and imperative paradigms; likewise an imperative mood (expressing commands) in most spoken/written languages. One might read Dothraki or Klingon, a brutal class of LCS languages and particularly “imperative” in their ‘commanding’ manner, unapologetic guttural articulation. But what might be the meaning of declarative? Do many people know? The internet suggests not. As per uszhe, everyone has his own definition, disambiguations + citation needed, wikipedia, hint hint.

So what’s the best language to communicate what we want, when the writing about languages is indecisive and muddled? Probably, and unsuprisingly, the language you speak best. True masters can adapt languages to their purpose, but most still recognize that CS languages are freighted with an intention, and this limits their applicability to all situations. The ambiguity of classifications like “declarative” in reference to a few languages or other terms applied to and restricting language adoption crumbles when you consider languages for their ideal operations, and not their syntax or semantics. What is the purpose of the language, how to absorb typical commands and how to appropriately pair them for more complex operations? Operation-oriented language selection (ruby is good for… and …) rather than grammaticentric (ruby syntax is “bloated and confusing“) might be the best approach for study; one that respects the romantic tropes of surprise, and pushes you to build a vocabulary based on the declared objectives of your goal, rather than the pretense of some predefined language hierarchy.

That, appropriately, perhaps unsurprisingly, is what she sed.

ilikethisNote: I like to alliterate my titles so if you thought this would be a post about functional programming and are now disappointed, you should check out my friend Jonathon’s post on functional programming coming out in Smashing Mag at some point in the soon, or this explanation series which is fairly brill IMHO.

If you wanted more stream editing and shell scripting, some resources you might enjoy are this one, and for awk reading (the best!), this one.

Tagged , , , ,

Fortune Telling with Data: Modeling Threat with Feeble Predictors

Anna Pavlova - foxy chiromancerWhile in college, and unbeknownst to most people, I dabbled in some performance artistry and predictive analysis. Part of the performative nature of college is developing the capacity to claim competence in topics and credentials one has yet to earn, to educate formally while informally fumbling through social mechanics for which no adequate prerequisite is ever published, and finally to make elaborate promises to potential employers you can’t yet keep. Corroborating this farce with some documentation is usually expected, a cover letter here, a résumé there.

On professional applications, it seemed impressive to have a range of acronym memberships to organizations with undefined but assumed-legitimate titles. Interviewers, however, seemed inattentive to merit or documentation fluff, so in small print among some legit scholarships and volunteer positions, I wedged in a nod to my extra-curricular involvement in the “ESP Volunteer Aid Org.”

esp

I am not a performance artist or a seer and this was an experiment I dropped shortly after, but I think there’s some irony in that, while attempting to assemble my prospective professional qualifications, I spent some time considering a career in heightened sensory perception. Or rather, I tried to jest-test the merit system with an obviously bogus acronym.  Funny in retrospect, but previous experience in ESP isn’t far from the tacit prerequisite many would assign to data mungers these days, and particularly anyone who does even mild statistical analysis on crisis datasets. With all these data, surely someone would be able to claim clairvoyance, solve international crises with a affinity for computational analysis of historic precedent, surely the answers are there?

psychic

fortunetellerAs someone who works with data, and more importantly as someone currently living in the future (compared to those currently living in my home country…whoohoo Nairobi time), I thought it might be appropriate to exercise my clairvoyance and provide clarity on life in Nairobi, assessing some of the current intensities, and the probabilities they might aggravate. In any case, the explosion of state department travel warnings in my inbox this week has made my reticence on the subject a bit obnoxious so I’m going to diverge from my typical open source software soap-boxing to write a bit about the statistics of terrorism and the particulars of my current condition.

I write from the position of a math hobbyist,  and an amateur clairvoyant, and Electric Powerso I’ve peppered this post with some specious but thoughtful observations about the news I’ve been following, what I’m currently experiencing, and the links, images, and resources my limited bandwidth allows me to explore.

In brief, Nairobi has been intense of late. The state department issued 4 official warnings this week, encouraging US residents and visitors to avoid Eastleigh, travel to Mombasa, proximity to Burundi, and most recently, travel in Kenya, period. These precipitate from the bombing earlier this week in Nairobi (6 fatalities and 20+ injuries), the recent church attack in Mombasa (4 fatalities) and general insecurity about al-Shabaab and armed operatives threatening attacks in any country conducting peacekeeping and/or military efforts in Somalia. :(

For it be morrow.. Speculation about the probability of a “large scale attack to come” made me start thinking about the meaning of “large scale” and projected “imminence” when it comes to statistically predicting events of high variability. How large is a “large scale” event? Anything where multiple deaths result seems “large” to me, though my definition has adjusted to accommodate recent conditions. If authorities are projecting a large-scale event to come, what about the unsettling events of now? How soon is imminence, not to be too much of a Morrissey fan-girl, but how soon is now?

Which to choose?So with all of these questions and my own preoccupation with quantifying self, I thought it might be time to read up on a few predictive models of the likelihood that something might happen.

Periodically people people post comparatives online like “you’re __times likely to die of x than get involved in a terrorist attack.” These were net popularized post-9/11 it seems, though, the scale and impact of that attack in the domestic US was fairly singular and data collected prior to it would do little to predict its occurrence and re-occurrence without admission of several limiting factors and uncontrolled variables.  That said, in the Annals of Applied Science (vol. 7, no. 4 2013) last year, Clauset and Woodward wrote a paper called “Estimating the Historical and Future Probabilities of Large Terrorist Events” in which they hoped to define a generic statistical algorithm for estimating the likelihood of terror events in complex social systems. These kind of predictive stats depend on so many variables outside precedent empirical data but the authors present multiple tail models and disclaimers about the limitations of their predictions to control for this (Matlab code and sample data available here if you want to play).

Kreskin's ESP board game

Of particular interest is their summary forecast are estimates for 3 potential scenario probabilities based on possibly forecast from past data:

“Rather than make potentially overly specific predictions, we instead consider three rough scenarios (the future’s trajectory will presumably lay somewhere between): (i) an optimistic scenario, in which the average number of terrorist attacks worldwide per year returns to its 1998–2002 level, at about ⟨nyear⟩ = 400 annual events; (ii) a status quo scenario, where it remains at the 2007 level, at about 2000 annual events; and finally (iii) a pessimistic scenario, in which it increases to about 10,000 annual events.”

That then, looks something like this:

table
Crystalball looking cuteThe Clauset and Woodard analysis further predicts a range forecast of 19-46% chance that at least one catastrophic global event will take place in the next decade. But to localize this a bit, and for the sake of argument, let’s take their “status quo” (a modest median) probability calculation trained from the RAND-MIPT Terrorism Knowledge Base, and set up the conditional probability that there will be a terrorist happening of catastrophic proportion (p=0.461) while I am in Nairobi (approx. 30/(365*10yr) possible days; p=0.008) . The condition is fairly unlikely, but unfortunately increases when you factor in covariates like my general foreignness (> victim likelihood…bummer, p=0.475), and the  logic that the violence occurring with agglutinative regularity will likely foster additional conflict and escalated tension:
“For instance, international terrorist events, in which the attacker and target are from different countries, comprise 12% of the RAND-MIPT database and exhibit a much heavier-tailed distribution, with αˆ = 1.93 ± 0.04 and xˆmin = 1.”
EAC MapTrying to control for multiple variables is complicated, so even in a problem which can be structured as conditional (probability of x given n state) struggles in this scenario. Does the probability of one state affect the other and yet still require factoring in both? And if so, perhaps it’s a joint probability issue between independent events. When the prediction derives from historical information, perhaps a Bayesian use of prior probabilities could be trained for future forecasting but even then…complicated. And regardless, perhaps the historical data is limiting in applicability due to scope; my definition of “catastrophic” scales down to the mere injury of a family member/friend, decidedly distant from the catastrophic proportion of 9/11 or any event with Chiromancerupwards of 1000 fatalities used to make these kind of probabilistic predictions.
Most of the math here is beyond my own research level but one factor that strikes me as strange, given my work with crises in the context of maps, is the absence of particularly specific geo-data analysis. The East African Community (EAC) hasn’t been spared much violence in the past few years, and sadly, in the last few months in Kenya. so I’m interested in reading about statistical modeling done on conflict probabilities with geo-specificity. Maybe this a usecase for the Wolfram Language when I’m brought out of my beta-in-waiting status; something to counter the pop-y around the world travel time estimates and polar auto-opposite calculations that have been so fun but maybe not particularly applicable to my current situation. What might be applicable is a computational knowledge engine that would assess my IP address, map it to a lat/long and then calculate how far I should move in the city to avoid conflict on a daily basis (*winks* at Wolfram friends).
To be fair, the Clauset and Woodward research  honestly nods to the variables not-completely considered in their analysis:
“Technology, population, culture and geopolitics are believed to exhibit nonstationary dynamics and these likely play some role in event severities…our approach is nonspatial and says little about where the event might occurrefinements will likely require strong assumptions about many context-specific factors (Clauset + Woodward, 15).”
Evangeline Adams (American Astrologer) explores a mapBut, I’m still wondering about alternatives to these estimations, what is the best research body to design these kinds of models and who has the best open test data on the topic? I’m generally skeptical of predictions based on historic data without geo-reference these days, since so much of what happens depends on a cultural/historical/social context that is impossible to divorce from a particular place;  the general forecast of 19-46% chance of something happening in the next decade at a global scale is hard to conceptualize when you consider the umpteen geopolitical factors that might cluster likelihood around certain high-tension locales (Clauset + Woodward 14). Perhaps there will one day be a service to prioritize these factors and co-variates based on personalized social and surveilled data as Seth’s Worry App concept suggests:
“Worry is the very first technological solution that maximizes the benefit of mankind’s oldest task: anxiety.
Using this flow of data, the Worry app computes the things you ought to be worried about. For example, instead of needlessly wasting time worrying about a random event like being bitten by a brown recluse spider, the Worry GPS system can point out that based on where you are, you’d be better off worrying about a different, unpreventable event like being killed by a fire hydrant flying through the air or perhaps by an angry rooster wielding a knife. The Worry app will alert you to that, which dramatically increases the effectiveness of your worrying.”
Destiny awaitsI’m into anxiety optimization and maximized thought efficiencies, perhaps a maturation of my adolescent ESP :)
In all seriousness, there is probably little statistical value in projecting these possibilities where I am currently. Fortune telling with so many variables can be complex, though the projections remain pretty unsettling.
But apart from all the speculative quant, there are some simple qualitative observations that I can make:
  • things are heating up every day here
  • any situation where “safety in numbers” is a paradox because avoiding congregation points (malls, churches, etcet) has become a way of avoiding conflictESPad is probs bad
  • at the end of the day, statistical randomness is a really unfortunate jerk who even despite your best precautions can allow for some pretty horrific happenings (a carjacking happened in my inner circle this week, for example)
  • there’s something broken about the fact that the entire reported anti-terror budget of Nairobi is less than my current apartment’s rent outside the city (both cases supposedly sustaining a month’s worth of expenses). If we drill down on that statement semantically, and not quite statistically, we can conclude that the collective safety of a city in a time of “imminent” crisis is roughly worth a one bedroom apartment.

Mathematical calculated? Feebly. Statistically significant? Probably. Totally unfortunate? Predictably.

Tagged , ,

Crowd-ed + Coordinated: FOSS in Africa

“There’s no more powerful force in modern society than the news. It shapes how we see the world, what we judge to be good or bad, important or silly, right or wrong.”
~ Alain de Botton, “Have you Heard the News?” Psychologies, 4/2014

In the April 2014 issue of Psychologies Magazine, Alain de Botton’s interview discusses his new book The News: A User’s Manual, and his philosophical reading of the news as trending toward more personal, more philosophically predictable. It’s perhaps significant that I’m reading this article in an airport news stand out of a pop magazine, rather than reading his book. More on this trend in abbreviated news ingest later…but for now, his points about our pot-boiler appetite for the news does well to introduce some of my recent professional happenings, perspectives on crowd-driven data journalism, and particular perspective on crowd-data programs in Africa.

Nairobi - Crowdmap of Tweets

In Nairobi, while the news has been of late focused on other topics, the last two weeks IDLELO Conference Badgesof my workflow concentrated on two conferences, a IDLELO: FOSS conference and a Global Innovation Competition for citizen-driven government initiatives; they share crowdsourcing and open journalism as themes. I had the pleasure of speaking at the IDLELO-06 conference, supporting Ms. Angela Odour’s talk on Ushahidi prior to preparing my own with James Raterno and Daniel Cheseret of Internews-KE. Of the few journalism organizations presenting, we applied the free-and-open-source-software (FOSS) theme to investigative news reporting and interactive political commentary. Our talk was a case study in health projects, demoing three interactive news stories from this past year at Internews-Kenya. Each interactive delved into some aspect of health monitoring in Kenya, spanning a spectrum of topics from medical services availability to mapping the outposts and effects of extractive industry across the country. While the details and data behind these stories are important and interesting, the presentation in each case was paramount; TL;DR the realities of healthcare and economic/industrial health of the nation were best communicated via interactive charts, and Internews’ series of Data Dredger infographics. The refrain of this and de Botton’s Psychologies perspective persists: attractive and interactive stories, stories that engage with personal, psychological topics, stories that illustrate rather than allude to data are driving our journalism programs and our teams.

Crowdsourcing Comic - XKCDAnd part of that means democratizing the newsroom to a broader population of citizen journalists and crowdsourced contributors, part of this also means broadening our view of where data journalism trendsetting is happening in our world, but to persist on these points, let’s move off the African continent briefly.  Among the most popular articles in the NY Times last year were approachable, interactive pieces; it’s not unreasonable to conclude that the appetite for news often bends to people’s visceral interests, regional perspectives and even “popular biases” as de Botton suggests in his Psychologies interview. Likewise, the Guardian’s 2013 popular titles for most popular articles (among Snowden and the Boston bombing coverage) include the following:

  • Why have young people in Japan stopped having sex?
    3.2m page views, 1,263 comments
  • Michael Douglas: Oral sex caused my cancer
    2.0m page views
  • Royal baby: Duchess of Cambridge gives birth to a boy – live
    1.5m page views

Global Innovation Challenge CrowdThis is not to suggest that the most popular news publications follow predominantly potboiler subject lines, but rather to note that there is a persistent appetite for pop culture throughout all news sources and dissemination platforms, irrespective of reputation. Mixed in with the seriousness and severity of crises worldwide, the presence of pop culture news commands significant attention; perhaps Global Innovation Challenge Collab - Nairobi, KEreflecting an appetite for popular and approachable media. When de Botton claims that “the ideal news would take into account people’s natural inclinations…it wouldn’t start with the wise, good, or serious outlooks,” I thought the judgement was a bit unfair and dismissive of journalism’s future, but maybe, on reflection, not so removed from reality in journalism’s present (Psychologies Magazine, 54).

This media appetite is agnostic to journalism hierarchies, persistently attracted to KE-MAVC8personalized stories, that show how one girl lives in NYC projects, or how a population’s accent differs according to regional divisions. We crave a personalized experience with the news even in the most distinguished publications, we crave a flat structure of open contribution, where the stories are interactive, where we can comment publicly in the thread following each post, where the content is sometimes crowdsourced, and the platforms are participatory. Our appetite for pop culture parallels publication output. In a digital media landscape where everyone from Buzzfeed to Fbook to O.K. Cupid have a data science team, our population of increasingly connected readers is interested in the personalized analytics of their networks, in the data science that drives our personal lives and pop culture as much as our professional publication platforms, and sometimes, in how all of these data fuse.Lagos - Crowdmap of Tweets

One way to adapt to this is to invite more contributors into the news reporting community from the reported community; to flatten the reporting structure, to amplify the data-driven projects that drive the page view counts often used to index our community impact. Promoting “popular” media isn’t just about echoing celebrity gossip and simplified story-lines but rather developing a sensitive authoring practice, crafting stories that readers can identify and interact with, and this trend is carrying into bootstrapped newsrooms across the African continent and throughout the world. In supplement to interviews, we crowdsource data collection in the way of Ushahidi, instead of lone-wolf work of an re-located investigative journalist, we train teams of indigenous journalists to report on their own local communities in the way of Internews. I’m privileged to work with organizations actively contributing to this type of globalized citizen journalism and crowd-reporting, likewise privileged to work with journalists when I am at best an “outsider-[FOSS]-artist.”

This is not new science of course, most established papers have a data teams these days, and it’s not uncommon for teams of developer-journalists to collaborate on investigative pieces, but to recognize the trends as reflective of an interest in crowd-driven projects, and citizen-journalism engagement globally is perhaps important and worth considering as we re-evaluate where journalism is, and where it is going.

Accra - Crowdmap of TweetsCrowd-sourcing information, crowd-funding and crowd-feedback loops in the journalism community are more popular, and not just in the USA. Analytics permit us to track what our crowd of readers actually reads (or at least what they click on), to adapt our stories and investigative practice to suit those interests. Though we still have a rockstar reporter hall-of-fame that celebrates individuals and their contributions to the industry, with data-driven projects, we can now appreciate more than ever, that often, and maybe always, the byline includes a team, a small crowd of developers-journalists-researchers working on a comprehensive and data-informed investigation.

“I doubt if it makes much difference, frankly, but at the margin I think that we’re moving to a kind of journalism that is more casual, more informal, more personal, and a very formal byline seems as out of place as a three-piece suit in the newsroom.”
~ Nicholas Kristof, “What’s Missing in my Byline,” New York Times: Opinion Pages, 1/2014

Tunis - Crowdmap of TweetsAnd this isn’t only happening at the New York Times or The Economist, it’s happening in Africa too. This brings me to the second conference happening of the past two weeks of work. At this week’s Global Innovation Challenge week in Nairobi, we’ve been working with teams of selected delegates from 10 countries around the world, teams who are working to connect their citizens more directly with their governments and foster policy change through open data. This type of effort can read as a quixotic ambition, but with developer and data-driven programs, it is possible. Johannesburg - Crowdmap of Tweets

Further, it’s noteworthy that all of the delegates are paired teams, not-lone crusaders, these efforts are built on partnerships between multiple contributors (developers, political activists) and multiple institutions, on crowd-driven programs meant to collect a maximum of opinion and surface a population of opinions from a representative sample of constituents. Supported by Ushahidi and hosted by iHub, this week of conference talks, pitches and programs is designed to foster more crowd and community driven data reporting across the globe, and model the crowd-centric trends so observable in our increasingly personalized and popular media.

Crowd-driven journalism and FOSS initiatives have in one respect opened the community to a broader population of self-taught developers and scrappy reporters, and also broadened the potential for citizen-sourced, -funded, -voted journalism projects. The crowd will doubtless drive even more data projects in the future, and craft a more personalized and popular media with a global scope. Crowd + Africa doesn’t have to mean crisis mapping or violence, it can mean participatory reporting and progressive reform, it can mean a program of re:activism, or react-ivism, piloted by a crowd of programmers and a ragtag group of pirates and outsider journo-artists. We’re working to amplify the crowd, and data-driven newsrooms internationally, in keeping with up the [journalism] Joneses.

Ushahidi Ecosphere Diagram

To that end, and in conclusion, I leave you with a link to our Ushahidi community survey, an effort on our part to make crowdsourcing a part of our own analytics and feature development workflow. Please fill it out so that we might improve our software and help other investigative journalists spin up custom instances of geo-local data collection all over the world:

HELP US OUT, FILL THIS OUT:

CROWDMAP COMMUNITY SURVEY

Recent Happenings:
Current:

Upcoming:

Images in this post courtesy of XKCD, IDLELO06, Global Innovation Competition, and FloatingSheep.org (African tweetmaps)

 

Tagged , , , ,
Follow

Get every new post delivered to your Inbox.

Join 29 other followers