Welcome to This Week in Modern Software, or TWiMS, our weekly analysis of the most interesting and important news, stories, and events in the world of modern software and analytics.
This week, our top story concerns the big data technology used to analyze and understand the Panama Papers.
TWiMS Top Story:
Panama Papers: Inside the Technology That Made It Possible to Tell the Story of The Biggest Leak in History—International Business Times
What it’s about: Data leak? More like a data flood. The internet went haywire this week after the reveal of the Panama Papers, a massive, 2.6 TB data leak from Mossack Fonseca, a Panamanian law firm that helps set up anonymous offshore shell companies. The leak, the largest ever of its kind, included a staggering 11.5 million documents—emails, PDFs, photos, and more. German newspaper Süddeutsche Zeitung revealed that it had been contacted via encrypted chat by a still-anonymous source more than a year ago, who subsequently began sending chunks of the data to one of its reporters, Bastian Obermayer. The paper began analyzing the data with the help of the International Consortium of Investigative Journalists—and modern software tools, of course. The team—comprised of roughly 400 journalists from 100 different media organization in 80 different countries—began publishing its findings this week, with far-reaching consequences and implications, bringing down world political leaders and roiling local real estate markets.
Why you should care: There’s still plenty we don’t know. For all of the coverage this week, the story and its fallout is just beginning to unfold. But reports indicate the person responsible for the leak accessed the data by hacking into the law firm’s email server, where messages were stored unencrypted. As the size of the data trove grew, the team of journalists poring over it needed help to make sense of it, store it securely, and keep it from leaking again until they were ready to begin publishing their analysis and reporting work. Forbes reports that the group used open source encryption software VeraCrypt to protect the data during its yearlong investigation. (International Business Times, meanwhile, tells the story of how a small software firm called Nuix gave the ICIJ team free access to its software for organizing large amounts of unstructured data—the Panama Papers certainly seem to qualify—in an indexed, searchable database with a “Google-style interface.” ICIJ’s developers also built their own search engine, with two-factor authentication, on top of the Nuix database, according to the IBT report. The Forbes account indicates the team used Apache Solr and Apache Tika open source tools. Integrated graph database technology from Neo4j and data visualization tools from Linkurious were also part of the toolset, according to ComputerWeekly.com. To make the data accessible to stakeholders across 80 different countries, Forbes reports that the ICIJ uploaded the data to Amazon Web Services, where reporters could access, search, and analyze it.
- About the Panama Papers—Süddeutsche Zeitung
- From Encrypted Drives to Amazon’s Cloud—The Amazing Flight of the Panama Papers—Forbes
- Latest Updates of Panama Papers Fallout—USA Today
- Data Leaks of the Rich and Famous: Panama Papers Show No One’s Secrets Safe—CNET
- Technology Has Made It Easier to Steal 11.5 Million Documents—USA Today
- Panama Papers Revealed by Graph Database Visualization Software—ComputerWeekly.com
- What’s the Deal With the Massive Panama Papers Leak?—CIO
What it’s about: Messaging service WhatsApp, used by more than 1 billion people worldwide, announced this week that it has rolled out end-to-end encryption across its app. WhatsApp, which Facebook bought for $19 billion in 2014, already encrypted chat messages. Now, it will do the same by default for any communications between users running the latest version of the app: “every call you make, and every message, photo, video, file, and voice message you send, is end-to-end encrypted by default, including group chats,” wrote co-founders Jan Koum and Brian Acton in a blog post. End-to-end encryption is just that, and even WhatsApp won’t be able to access a user’s communications: “When you send a message, the only person who can read it is the person or group chat that you send that message to. No one can see inside that message. Not cybercriminals. Not hackers. Not oppressive regimes. Not even us.”
Why you should care: In light of this week’s Panama Papers and last week’s Apple-FBI denouement, this one probably doesn’t need an explainer—but here goes: As CNN Tech’s James Griffiths notes, rolling out end-to-end encryption to its entire user base immediately makes WhatsApp “the most widely used cryptographic tool on the planet.” Indeed, it puts full data encryption at the fingertips of anyone with a smartphone—which is why the rollout is being praised by privacy advocates and others, but has already drawn attention and comments from the FBI’s top lawyer. The big deal here though, writes Fortune’s David Meyer, is that “Users don’t need to consider the arguments about having ‘nothing to hide’ and the balance between national security and privacy…. This is now simply what their communications app of choice does.” Still, observers like SecureMySocial CEO Joseph Steinberg point out that the move doesn’t make WhatsApp totally private and secure or, as Steinberg puts it, “warrant-proof.” That’s because the metadata associated with WhatsApp communications still won’t be encrypted—and unencrypted metadata is a key reason why a recent Berkman Center report found that it’s unlikely criminals and other bad actors will actually be able to truly “go dark” online.
- Top FBI Attorney Worried About WhatsApp Encryption—U.S News & World Report
- Forget Apple vs. the FBI: WhatsApp Says It Has Enabled End-to-End Encryption for All Forms of Communication on Its Service by Default—WIRED
- Why WhatsApp’s New Encryption Is a Game Changer—Inc.
- WhatsApp and the Backdoor Battle—BBC News
- WhatsApp Adds End-to-End Encryption for All Communications—CNN Tech
What it’s about: The FCC this week unveiled new information labels for home and mobile broadband that resemble the nutrition labels found on food packaging. (Seriously: The sample labels available on FCC’s website look almost identical to what you’d see on the side of a box of cereal.)
Too often, advertised download speeds seem like ambitious daydreams instead of reality, and the labels are intended to give consumers clear, accurate information when comparison-shopping ISPs and mobile carriers for both cost and performance. (There are separate labels for landline and mobile networks.) Information includes “typical” download and upload speeds, pricing, one-time fees, data allowances (and the costs of exceeding those allowances), hardware leasing costs, and other data.
Why you should care: With so much of modern software dependent on fast, reliable broadband access, transparency from data providers is good for consumers and the software industry. Not surprisingly, there’s also plenty of skepticism, given the often fierce opinions on net neutrality and, more generally, the government’s role in regulating the internet. (And, of course, consumers still have to read and understand the labels for them to do any good.) There’s a more specific rub, too: ISPs and mobile carriers aren’t actually required to use the new labels, though doing so would “satisfy the FCC’s requirement to make transparency disclosures,” according to the FCC. New rules requiring those transparency disclosures, regardless of whether providers use the nutrition labels, kick in later this year. The FCC tells Ars Technica the new transparency requirements will stand even if its 2015 net neutrality order is eventually overturned or altered.
- FCC to Unveil Broadband Nutrition Labels—Politico
- FCC Creates Broadband Nutrition Labels for Consumers—InformationWeek
- The FCC’s New Broadband Explainers Just Make It More Complicated—Gizmodo
- FCC’s New ‘Nutrition Labels’ for Broadband Services Leave Out a Few Ingredients—Yahoo! Tech
- Consumer Labels for Broadband Services—FCC.gov
What it’s about: It’s no small job making sure Google’s slew of super-popular apps, from Gmail to search to Maps and more, are always available. A new book from O’Reilly, Site Reliability Engineering: How Google Runs Production Systems, explains how they do it—right there in the title. “Site reliability engineering” (SRE) turns out to sound a whole lot like DevOps in that it tries to overcome the traditional barriers and conflicts between development and operations responsibilities and skill sets. But, as WIRED’s Cade Metz reports, DevOps “evolved separately from and largely after the SRE philosophies that arose inside Google.” It’s just that Google didn’t talk much about SRE—until now.
Why you should care: For starters, Metz calls Google’s new book a “must-read” for anyone interested in DevOps. Even just the preface, introduction, and first chapter provide “a fascinating look at the attitudes that drive the world’s largest online empire.” Chapter one is authored by Ben Treynor Sloss, Google’s VP for 24/7 operations who’s credited with the term “Site Reliability Engineering” and the approach’s growth within the company. Even Sloss’ initial hiring speaks to the DevOps connection: He was a developer hired to run operations. That might make traditional ops folks cringe, but it shouldn’t: Sloss calls site reliability the “most fundamental feature of any product,” according to Metz. So how did Google bring down the longstanding wall between dev and ops? Sloss cites the concept of an “error budget,” according to Metz: 100% uptime is not actually the goal. “An outage is no longer a ‘bad’ thing,” Sloss writes. “It is an expected part of the process of innovation, and an occurrence that both development and [operations] teams manage rather than fear.” Think of it as a “stretch goal.”
iPhone Cracker—The Daily Mail
What it’s about: A weekend story in U.K. newspaper The Daily Mail that ran with a hyperbolic headline—we’re using a condensed version above—suggests that the FBI could have unlocked the iPhone used by the San Bernardino shooter simply by buying an inexpensive device called the IP Box on eBay or elsewhere online. The paper purchased one of the devices for about $170 at a brick-and-mortar mobile store in a Sheffield, England, Fone Fun Shop and hooked it up to an iPhone 5C. Six hours later, the device had cracked the iPhone’s 4-digit passcode. While the iPhone will normally disable access after too many incorrect passcode attempts are made, The Daily Mail says the IP Box is somehow able to keep entering different codes until it finds the correct one. “What took the FBI weeks can apparently be done in a matter of hours with devices like the IP Box,” the story contends.
Why you should care: There’s probably more than one reason why this story didn’t make bigger waves on this side of the Atlantic. For starters, as multiple comments on The Daily Mail’s website point out, a 4-digit passcode isn’t particularly sophisticated. (Use strong passwords, people.) And the iPhone 5C that the paper cracked was running iOS 7, which is now two full versions old and lacks iOS 9’s security enhancements. Indeed, it was not a particularly high-tech hack—just a brute-force password attack with a device that looks like a leftover prop from the 1983 movie WarGames. The device started at “0000” and kept trying until it found the right combination, in the test case “3298.” Still, it’s important to remember that no device is completely secure, and locally stored, unencrypted data is always at risk, especially with weak (or no) passcode protection. And while the phone that The Daily Mail cracked might have been especially susceptible given the 4-digit passcode and outdated operating system, device vulnerabilities don’t end there. Fone Fun Shop’s company director Mark Strachan claims in the story that it will begin selling a new device later this month than can crack passcodes on Apple devices running iOS 9.
What it’s about: A new photo-and-interview project called “Techies” has become the talk of, unsurprisingly, a lot of techies this week. Creator Helena Price describes the project’s two primary aims on Medium: “I wanted to show the outside world a more comprehensive picture of people who work in tech. I also wanted to bring a bit of attention to folks in the industry whose stories have never been heard, considered, or celebrated.” In an industry with an well-publicized lack of diversity, the Techies project takes a different tack: Shining a spotlight on people who, as the project’s website reads, “tend to be underrepresented in the greater tech narrative. This includes (but is not limited to) women, people of color, folks over 50, LGBT, working parents, disabled, etc.”
Why you should care: “Perhaps you’ll use it as an opportunity to learn more about people from different backgrounds than you,” Price says on Medium. “Perhaps you’ll find people who finally look like you, come from where you come from, or have been through similar struggles.” The Techies homepage includes a grid of the project participants’ portraits that can then be filtered by various identities, such as “First Generation,” “Developer,” and “Trans/Genderqueer.” Price points out in an interview with TechCrunch that the term “techie” itself has turned negative: “Now people hear it and they sort of cringe. Everybody has an image in mind of what that word is. For them, to see that word, expect one thing, and then see a grid of images that don’t look like what they expect is an experience in itself.” It’s an eye-opener—be sure to check it out.
Want to suggest something that we should cover in the next edition of TWiMS? Email us at email@example.com.
Tune In to the Future
Can’t get enough modern software news and commentary? Be sure to check out our Modern Software Podcast. New Relic Editor-in-Chief Fredric Paul and guests discuss the most important things happening in the world of software analytics, cloud computing, application monitoring, development methodologies, programming languages, and more. Listen to episode 10 or subscribe on iTunes.