Olympic Intelligence Gathering

An investigative article I have been working on was just published in the Dominion. The main themes are looking at the US side of Olympic security, the role of Multinational corporations in intelligence gathering, the emergence of Fusion Centers in the US and the in roads fusion centers are making in Canada.You can find a link to to the story here.

I have been doing investigative research for many years now, but I am only now learning how to apply that to journalism. One of the dilemmas I faced in writing this article was how to tell a very complicated story in 1200 words. I think in retrospect it might have made more sense to do a series of articles and fully hash out each idea.

the costs of investigative reporting

At the end of August the Mother Jones published an interesting article that looks at the high cost of doing investigative reporting. They focus on a story that was recently published in the New York Times Magazine, written by a Propublica staff writter. The investigation took two years, and cost nearly $400,000.

They write:

This story... is the kind of work that is in peril now that the financial underpinnings (i.e. advertising) for journalism have collapsed. Bloggers and commenters and citizen journalists can’t take on a project like this. They can add to it, amplify it, criticize it, and generally run with it, but a project like this requires consistent, institutional teams of reporters and editors and factcheckers and lawyers and web dudes.

Building sources of funding for investigative reporting will be a challenge going forward.

[thanks Ben]

Get off the net

So far almost every post I have made on this blog has talked about Internet. To be clear my view is that to do good research you should never limit yourself to looking on the Internet. Google does not have all the answers, and neither does the deep web. Of course the phone is probably one of the most important tool out there, but this post is about old book you find in the library: City Directories.

City Directories are books that list who lives in a city. The older versions list people by name and address, later they added phone numbers. In recent decades they have morphed into phone books, which only list people by last name. If you want someones current phone number, or to find who lives at an address of phone number today you can use a 411 site. But I have never found an online service that lets you look at what someones address and phone number was in your year of choice. To do that you have to visit the library.

City Directories are usually found in the genealogy collection. They won't be at every branch, but somewhere in your region should have them. Every year the library will get a copy of the phonebook, and they often have volumes of the city directories going back to the 1800's. These books are heavy and grabbing a decade worth of books off the shelf and bringing them to a desk might take several trips.

By going through the books you can find out not just where
someone lived but what years they moved in and out of an address. All you have to do is check the years on either side, until you find the year when their address is not listed.

Of course this technique has some limits, people change their name, only one person in a house has there name in the books, and nowadays cell phones and even many landlines do not get listed in the phone book. But I have used this technique a handful of times and found it real useful. Some times it was for historic research, but on other occasions it was for research about someones address in the last couple years.

Halifax history and crowd sourcing

I have spent the last two week in Halifax, making history posters about Halifax history, and wheat pasting those posters up in area the posters talk about. I have been doing a residency at the Roberts Street Social Centre, working on this project. It is based on a project where I have been doing the same thing in Toronto for many years, the Missing Plaque Project. I thought this would be a great moment to talk about a research technique I used to help with one of the posters I made a few years ago in Toronto.


A friend gave me a write up about a Halloween Riot of that took place in 1945 in the Beaches, a neighbourhood of Toronto. I now think the number was a little inflated but the article says there were 7000 youth rioting. It seemed like a great topic for a poster, I began researching it. After the riot a few newspaper stories had been written in the Toronto Star and Toronto Telegram. They did a fairly good job of explaining what happened at the riot, but did not explain the context. I was sure I only had part of the story. Despite exhaustive research I could not find any information that helped me figure out why youth in that era were so ready to riot.

My solution was to put up the poster anyways, but I included a paragraph saying I was sure i did not have the whole story, and asking for anyone who had more information to get in touch with me. I did not know the term at the time, but what i did was crowd sourcing. Basically crowd sourcing is putting out an open call to an undefined large group of people, in this case looking for more information. It is a great research technique.

By putting up posters in the area the riot had taken place I was targeting people in that area. Crowd sourcing isn't usually done through posters, but regardless of the medium, the principles are generally the same.

To finish my story, I got contacted by two people. One of them was a friend I was showing my posters too. It turns out he had written a play about street gangs in the 1940's and 50's. He helped me understand how youth who were too young to fight in World War II were living in the shadows of the returned veterans, with no chance to be Heroes and little chance of getting a job, since vets were given preference. I was also contacted by an old man who had been in the riot. He saw my poster and emailed me. We had coffee, and he helped me understand what the thinking of the youth was at the time of the riot.

You can follow this link to read the text of the poster.

NEW TOOL ON BING

When I first looked at Bing, (the new brand for Microsoft's search engine), I wasn't very impressed. I had always liked some tools under Microsofts old brand "Live". "Live Image Search", and "Live Map Search" are tools I liked better than Google's equivalent tools. Both of these have been carried over to Bing. However in terms of search results I wasn't being won over. Now a recent post on google blogoscope, is making me look twice.

The tool lets you search all the domains registered to an ip address. All you do is type ip: followed by an ip address. Combined with Whois this very useful in profiling a website and finding its connections to other sites.

I tried it out quickly with www.dominionpaper.ca. I used the tool from the post to find out the IP address is 209.44.112.66. When I looked this up in Bing the results showed me all sorts of sites sharing that IP address. A quick look showed me they were mostly media, civil and activist groups based in Montreal. The number of results was quiet large, 74,800, that is because the results are not only domains but also pages from within those domains. By plugging results into a whois search I found that Koumbit.net seemed to be the host server's name.

A very useful tool!

Ironically it turns out this tool is not new but had existed in Live search as well. I am certainly going to give Bing another chance.

+ syntax

The plan was to put up at least one blog post a week, and I will step up to the plate real soon with some great posts. I know i have been doing lots of Google syntax posts, here is another, i promise some more variety soon.

The Plus syntax, + , tells Google that you really meant what you typed in the search bar. it was really useful when google ignored one and two letter words. Now a days I don't use it very often at all, but it is good to be familiar with how it works.

Google likes to think it is smarter than you, (and it often is). Google will sometimes automatically change your spelling if they think you made a typo. Googler Matt Cutts explains this in more detail in a post on his blog. So if you typed the word correctly, and it really was what you were looking for you can tell that to google by adding a plus directly in front.

The other thing that Google does is include associated words, for example, if you search for motivates, Google may also search for motivated or motivate. and if you search for Street, Google will also included results for St. , Again, by adding the plus in front of the word you can tell google what you really wan; +motivates.

So here is an example of how I recently found this useful. I was researching Panis, who were natives enslaved by the French. This is what my search looked like:


Google assumed that what I really meant was the far more common word, Panic, and made that change for me. I changed my query to Slavery +panis, and then got the results I was after.

Note: you have to put the Plus directly in front of the word.
+ this + is + wrong +usage
+but +this +is +correct

SEARCHING STREET ADDRESS RANGES

Google has a peice of syntax that lets you enter number ranges into a search by putting two dots between two numbers. For example 2007..2011, will search for 2007, 2008, 2009, 2010, or 2011, if any of those numbers appear in a page it will be returns in the search results. This can be useful when searching for money ranges, eg. $40..$120.

However i didn't use this syntax very often, until I recently discovered that it works within quotes. That means if you are searching a municipal website you can put a range of numbers before the name of a street, and find any mention of those addresses on the site. This is great when you are researching not a specific address but rather a strip, or block with numerous street addresses.

Here are a couple of examples of what the search might look like:

site:phila.gov “4800..5300 Baltimore”

or

site:toronto.ca "800..1200 Bathurst"

Note: If you add St. or Ave. to the end ofthe street name you will not get all the results since you will miss the full spellings, Street, and Avenue, and visaversa.

ChangeDetection

Changedection.com is a site that lets you monitor a webpage to see if it has changed. If it does change you get an email. Ideal for tracking a site that doesn't have a RSS feed. I got tipped off about this site a few week ago at the IRE confernce. I tried it out and have been impressed.

After signing up for an account, I entered a the url for the site I wanted to track. I then got brought to a page that gave me a lot of options; Do i want to be notified only if information is removed from the page? Or if it is added? How often to receive notification? Most importantly they let you enter a word that must appear on the page before you are notified.

When the page changes you get an email that contains a link to the changedetection site, where they show a summary of the new information (highlighted) and the removed inforamtion (crossed out).

I haven't had a great need for this tool yet, So far the most useful way i have put it to use is on the websites for annual conference. Now if they add the word '2010' I will find out and get a head up on the when they update their website with information on next years conference. However I am certain this tool will come in real handy in my research at some point.

TASER DATABASE

The CBC and Canadian Press colaborated on a great investigation into taser use by the RCMP. They used freedom of information requests to get a hold of incident reports that officers are required to file when ever they draw their taser. On the CBC website you can find extensive coverage including a list of taser related deaths and a search tool that lets you search the data they compiled.

Their searchable database lets you search during specific years, by province, by the type of incident, and by the number of cartriges/spin cycle used by the officer. You then get a list of reports and can read the details from the report.

In a quick look I found that there were 23 recorded times when eight or more stun and cartrige cycle were used since 2002. And That there were six recorded incidents of tasers being used on cell blocks.

On the CBC website they write:

“By putting a searchable version of our database online, we hope that people can discover a few more details about the use of a device that has been the subject of increased public scrutiny and political debate after the death of a Polish immigrant, Robert Dziekanski, who died shortly after being stunned five times with a Taser at Vancouver International Airport in October 2007.”

check out the web site here: http://www.cbc.ca/news/interactives/taser-deployment/

Tweeking Google URLS

To get the most from a google search you can tweak the URL of your searches.  This post looks at how to have the date a page was created, and longer snippets show up in your search results:

It was recently pointed out in a post on lifehacker. They got the idea (from digital inspitation), that you can find out the date that google thinks a page was first uploaded by cutting and pasting this line to the end of your URL:

&as_qdr=y15

your searches should now have dates in front of the snippets. it should look like this:

This got thinking about one of my favorite feature in the new googles search options, the ability to get longer snippets in your search results. (that means more text from each pages being displayed in the search results). Playing with the URL I discovered that instead of clicking through to this option, you can simply cut and paste the following to the end of the url of your google searches.

&tbs=sts:1

If you are using google to do research, these two tweaks will help you get to the really good stuff. And, if you want to get these options added to your searches automatically, you can cut and paste the following link into you browser, and bookmark it (or make it your home page.) 

www.google.ca/search?q=example&hl=en&sa=N&output=search&tbs=qdr:y15,sts:1

It should give you a search for the word "example", but if replace it with a new word the added features will remain. (i have played around with this a bit, but if it doesn't work for you please let  me know.)

WAYBACK MACHINE - PART 1

The Wayback Machine is a tool that lets you look at old copies of a website.

 Want to see what city council was saying 2 years ago? Want to see what the google home page looked like in 1998? Want to browse a web site that has gone off line? Want to find old copies of a document that was posted online?     ...Then this is the tool to use.

Just go to Archive.org. enter a URL in the search bar, and click the "take me back button". you will now have a page with a list of various dates. click on a date and you will see what the site looked like on that date, and from there you can browse around. 

The Wayback Machine sends robots all over the web taking snap shots of pages, in the same way that google does. While google tries to get the most rerent version of a site, throwing out old copies, the wayback machine on the other hand holds on to these copies, and lets you see old versions.

If you want to learn more about the Archive .org, the group that runs the wayback machine, then check out the following  20 min video. There is only about 1 minute that covers the Wayback machine (at 17:30), but it give sense of what the broader project of Archive.org is about. check it out here: 

I will soon put up some more posts on the way machine that look at how to make it useful, and what advanced search features you can put to use to get the most from this tool...

Crime reporting in Baltimore sums it up

A friend sent me an amazing article that ran in the Washington Post entitled In Baltimore No One Left to Press Police. It is by David Simon the man who created one of the best TV series ever made, The Wire. In the article he reflects on when he worked as a crime reporter and how much things have changed since then.  He use the case  of a police officer shooting an unarmed man that went nearly unreported in the baltimore sun. 

In the Article he writes:

There is a lot of talk nowadays about what will replace the dinosaur that is the daily newspaper. So-called citizen journalists and bloggers and media pundits have lined up to tell us that newspapers are dying but that the news business will endure, that this moment is less tragic than it is transformational.

Well, sorry, but I didn't trip over any blogger trying to find out McKissick's identity and performance history. Nor were any citizen journalists at the City Council hearing in January when police officials inflated the nature and severity of the threats against officers. And there wasn't anyone working sources in the police department to counterbalance all of the spin or omission.

I didn't trip over a herd of hungry Sun reporters either, but that's the point. In an American city, a police officer with the authority to take human life can now do so in the shadows, while his higher-ups can claim that this is necessary not to avoid public accountability, but to mitigate against a nonexistent wave of threats. And the last remaining daily newspaper in town no longer has the manpower, the expertise or the institutional memory to challenge any of it.

My first reaction was that he summed up the the situation bang on. My view is that if you are a citizen journalist (or professional) this should be a call to step up to the plate and get serious about what you do, learn the technique, dig, prod, and be tenacious. 

More recently I saw a video of David Simon speaking that aired on democracy now.   

I think this gave me a better sense of what he was getting at.  He thinks journalism is a proffesion not a hobby. I would like to think that there are enough people out there who can do deepdigging and solid journalismwith passion as their main motivator, but it is true that finding the time and institutional memory is as large a challenge as devloping the nessesary skills.  futhernmore how do you get access to regular and costistant audiance.

I still feel that commited citizen journalists can step up to the challenge of doing seroius journalism. But there are a lot of questions Davis Simon poses that I don't have ansers for. What is clear is that anyone who wants to have journalism flourish in the furute has their work cut out for them.

Using Google Syntax: the Jeopardy technique

Here is a quick bit of Google syntax. A few months ago I was sitting artound with a couple friends, and we asked how many employees do you think Facebook has? One friend turned to Wikipedia, but did not find the answer. I suggested another approach that I have now come to call the Jeopardy Technique. When you have a question to do with facts, where the answer involves numbers, you state the answer to your question minus the bit of information that you do not have. You combine two pieces of syntax to make this work, the quotes ( “ ” ) and the wild card ( * ). In this case we used:

“Facebook has * employees”

Very quickly the answer came up in the snippets of the various search results, I think at the time the result said about 700 employees, but a more recent search shows an article written on April 1^st that says “Facebook has almost 800 employees”, while another article from March 11^th says “Facebook has nearly 1000 employees”. It is not an exactly clear which is most accurate, but it gives you a general range.

Of course you also get results that don't really apply to your search like “Facebook has helped corporations bolster employee morale ” or “Facebook has postponed its employees’ stock sale”. However if you add a third piece of syntax, the number range ( .. ) you can eliminate most (but not all) of the unwanted answers. The search will now look like this:

“Facebook has *..* employees”

or like this alternately like this:

“Facebook has * 1..1000000 * employees”

If you want only numbers and no additional words you can try this:

“Facebook has 1..1000000 employees”

Here are another couple examples:

-a search for “Jupiter has * moons” gives you a variety of answer including. 'Jupiter has 4 moons”, “Jupiter has four large moons and dozens of small ones” and “Jupiter has 63 moons”.

-A search for “Shakespear wrote *..* plays” provides the answer of 37 plays in the second and third results, while in the fifth result it gives the answer of 39 plays.

-In a search for “there are 1..100 species of penguins”, the vast majority of results state there are 17 species of penguin, one result claims there are 18 species and on 21 speices. For a long time penguin biologist did think there were 17 species of penguin, but more recently many of them have been arguing that one of these species should actually be divided into two separate groups. So 17 and 18 are both answers have truth in them. As for 21 species I am not exactly sure, where that number came from.

This technique works pretty well for finding the answer to questions where the answers are numeric facts. Of course if the answer is not a number this can be done even more easily, since you will need only the quotes and not the wild card. Just write the answer minus the last set of words. For example “the capital of Senegal is” ...Dakar, 'the fastest mammal is” ...The chetah, or “the CEO of Starbucks is” ...Howard Schultz.

Hopefully some of the techniques can come in useful to you. but it is also ment to be an example of how you can combine syntax in interesting ways. It should get you thinking creatively about how to put syntax into action.

THE NEED FOR INVESTIGATIVE CITIZEN JOURNALISTS

THE INTRO

The internet is profoundly changing the world of journalism in many ways. Given two of these changes, the rise of citizen journalism and the declining support from media institutions for investigative reporting, it is important for citizen journalists to develop deep digging research skills. This will allow them to help spread and increase the amount of investigative journalism being undertaken.

Investigative journalism cuts through the spin, covers important stories that would otherwise never be revealed, and finds lapses in other reporting. By going the extra mile to bring understanding to what is happening in the world, this genre of reporting is a vital part of the media. Without it, the news is merely a mouth piece for the most powerful in our society. It is important to find ways to make sure investigative journalism survives and flourishes.

Organizations are being built that try to make investigative reporting sustainable. They are building charities and foundations that can fund deep digging reporting by talented professional journalist. Yet dwindling funding is only one part of how journalism is changing. Institutions, formal and informal, need to be built that can help bring investigative skills to not just professionals, but citizen journalists as well. This article will look at what has brought us to the situation we find ourselves in today, and ask how citizen journalists can become investigative reporters as well.

THE IMPACT OF THE INTERNET

In several compounding ways the internet has destabilized the newspaper. The most prominent change is that in a world of Craigslist, Kijiji and Google AdSense, the classified ad is dying. Classified ads once provided a significant portion of a newspaper’s revenue. Another loss of revenue stems from more and more news being read online. Subscriptions to newspapers are dropping, and advertisers reach fewer people by placing an ad in the paper; online advertising can’t raise nearly as much money as print can.

With less funds coming in, newspapers spend less money on unique content, such as investigative reports, and pick up more pre-made stories from news wire services such as Associated Press, and Reuters. This solution may play well in the short term, but in the long run it just further undermines the newspaper’s reader base. Online users have no reason to read a particular newspaper’s stories if they are just copies of what dozens of other newspapers are printing. The value to readers of unique content is rising, just as newspapers are spending a decreasing amount upon it.

Broadcast journalism has so far been less affected by the rise of the internet. But as user bandwidth continues to increase, the changes that have hit newspapers may well affect broadcasters in similar ways.

THE ATTACK WAS ALREADY UNDERWAY

Long before the rise of the internet, media consolidation, corporate business models and massive lawsuits were already leading to an attack on investigative reporting. In the 1990’s large corporations began suing media companies that revealed damaging stories, for hundreds of millions- even billions- of dollars. Some lawsuits were based on reporting which was arguably incomplete, unethically obtained or done in questionable ways. At other times the mere threat of a billion dollar lawsuit was used to silence the reporting of legitimate investigations. The owners and managers of media conglomerates preferred to settle law suits rather than risk a significant loss of funds, even if it meant backing down on important stories.

In an article entitled Investigative Journalism Under Fire, Marisa Guthrie eloquently points to several examples of media companies backing down to threats. She describes a “watershed moment'' in which the tobacco company “Philip Morris sued ABC News for $10 billion over a 1994 report...that uncovered the then-shocking evidence that cigarette manufacturers manipulated nicotine levels to keep smokers hooked. The network, which was being acquired by Disney at the time, declined to fight Philip Morris in court and instead settled for a reported $17 million; it capitulated in a widely derided on-air apology...”

As these massive lawsuits became more common, corporate leaders saw that putting money and resources into investigative reporting did not have good cost benefits. The prestige that might come from putting out ground breaking news reports, clearly did not out weigh the risks of losing money, at least in the eyes of those making the decisions.

NEW MODELS EMERGE

There are many groups working to find new ways to fund investigative journalism. Most notable are nonprofit groups like theCentre for Investigative Reporting, The Pulitzer Center on Crisis Reporting, Center for Public Integrity and ProPublica. These groups seek out money from foundations and private donors, and use the money to pay a staff of investigative journalists. They then team up with existing media institutions like NPR, The Nation, and 60 Minutes, to co-produce stories.

Another model is the one taken on by the Danish consumer watchdog group DanWatch. They also collect funds to hire journalists, but instead of co-producing stories, they investigate stories that match their mission statement, and then create media packages on them, which often get picked up by groups like the BBC. They operate with an understanding that journalists are over worked, and are more likely to cover a story if it is accompanied by photos, videos, and audio.

Both of these models respond to the funding crisis that is hitting investigative reporting. However, creating new sources of funds responds to only one of the ways that the media is changing.

THE RISE OF CITIZEN JOURNALISM

The ways journalism is affected by the internet go far beyond tearing down the old funding models that journalism depended upon. One of the biggest changes is the rise of citizen journalism: Bloggers have come to see themselves as an important part of the media; Independent journalists have found new platforms to let the world see their reporting; And the growing ranks of laid off newspaper writers haven’t all given up on their trade, many of them continue reporting on the internet. A wide variety of people are taking up the mantel of covering the news.

On one hand citizen journalists are unpaid, and lack the credibility and reach that many professional journalists have. They often don’t have the skills and resources that their paid counterparts have at their disposal. But on the other hand, they are fueled by passion, they don’t have the same confines that paid journalists do, and they are powerful in their decentralized nature and sheer numbers.

It would be tragic if professional journalism ever withered away, but citizen journalists are here to stay. Their ranks and influence will only grow. Professional journalists should find ways to collaborate and work with citizen journalists in uncovering the news.

BUILDING SOMETHING NEW

How will investigative reporting fare in an age where citizen journalism is taking a larger and larger roll? What has been created to make sure investigative reporting spreads and flourishes in an age of citizen journalism? Institutions, formal and informal, should be created that can step up to the challenge of developing skills and talent in non-professional journalists. This will allow them to conduct in-depth investigations and communicate their findings effectively to large numbers of people.

Many investigative journalists came out of the newspaper world learning from the daily experience of digging into leads, and having to put out a story everyday. As the newspaper withers, other ways to develop these talents must be developed.

There are any number of ways to build and spread the skills that are important in investigative reporting. These could include mentorship programs, web tutorial videos, training organizations, networks, and resource websites. There are dozens of concrete skills that can be developed from using the telephone and the internet to dig up hidden information: from to filing freedom of information requests, and crunching information in databases; the list could go on and on.

The time is ripe for citizen journalist to be given clear routes to learn these skills and develop talents in investigative reporting. Finding ways to make sure that investigative journalism spreads in the coming decades, rather than declines, is important to our world.