Adam Caudill

Security Leader, Researcher, Developer, Writer, & Photographer

On the need for an open Security Journal

The information security industry, and more significantly, the hacking community are prolific producers of incredibly valuable research; yet much of it is lost to most of those that need to see it. Unlike academic research which is typically published in journals (with varying degrees of openness), most research conducted within the community is presented at a conference – and occasionally with an accompanying blog post. There is no journal, no central source that this knowledge goes to; if you aren’t at the right conference, or follow the right people on Twitter, there’s a great chance you’ll never know it happened.

There are many issues that this creates, here I will cover a few:

Citing Prior Research #

In most conference presentations, seeing prior research cited is the exception, not the rule; this is not because the presenter is trying to take all of the credit for themselves, but a symptom of a larger issue: they are likely unaware of what’s been done before them. Unlike other fields where it’s clear that research builds on the work of those that have come before, in information security, the basis of research is unclear at best and totally lost at worst.

This leads to work being recreated, as happened recently in the case of TIME / HEIST – where the same discovery was made and presented by two groups nearly four years apart. In this case, for one reason or another, the HEIST researchers were unaware of the TIME research, and presented it as a new discovery. This clearly demonstrates two major problems:

  • Researchers aren’t seeing prior research.
  • Research is not being published in a way that makes it easy to find.

When Brandon Wilson and I were working on verifying and extending the BadUSB research, we were clearly aware of the work done by SR Labs and clearly disclosed the role their work played in what we released – what we should have cited though was a number of others that had performed research on similar devices, such as the work on SD Cards, though we weren’t aware of it at the time we began our work. In this case, there’s a blog post and a recorded talk (which is far better than most others), though it’s still not something we had seen.

By not citing prior work, we not only deny credit to those that had moved the state of the art forward, we continually reinvent the wheel – instead of building on the full knowledge of those that came before us, we are recreating the basic results again. Instead of iterating and improving, we are missing the insights and advantages of learning from others, we are recreating the same mistakes that had been solved in the past, we are wasting time by rediscovering what is already known.

There is also the issue of finding the latest research on a topic – when sources are properly cited, it’s possible to follow the chain to founding research, and to the latest research, as this is so rarely done in work produced by the community, it’s impossible to find the latest, to see the impact of the research you do, or see what’s been done to validate research. By not having these connections, a great deal of insight is completely lost.

This is also a very significant issue for those performing academic research – as it’s considered misconduct to not cite sources, yet without a way to clearly identify existing research, it’s difficult to impossible to cite the work that the community and industry does. This furthers the gap that exists between academic and applied information security. Some criticize academic researchers for being out of touch with the rest of the world – a major part of that is that we make it impossible for them not to be.

Losing History #

Perhaps the greatest cost of not having a central knowledge store is that much research is lost completely – the blogs of independent researchers are notoriously unstable, often disappearing and taking all of content with it. We are sometimes lucky that the content is reproduced in a mailing list or been archived in the Wayback Machine, though in too many cases it is truly gone.

Countless hours are invested every year, and there is at least one conference every week of the year – with material that may never be presented or recorded again. Only those that attended are exposed to it, so it exists only in the memory of a few select people.

There was a time that a person could go to enough conferences, read enough blogs, follow enough mailing lists to keep up with the majority of new research – those days have long since passed. Today, it is impossible for any individual to remain truly abreast of new research.

Steps Forward & Backwards #

In the past, zines such as Phrack could help share tha great deal of knowledge that’s produced, though now with years between releases, it is far from able to keep up. An effort that was a real step forward, PoC||GTFO, has helped some – with a few issues per year and has been able to issue the collected papers from conferences. Though the highly humorous tone, irregular schedule, and level of effort required to release a single issue bring up questions of suitability for the solution to this problem.

The Proposal: An Open Journal #

On August 21st I tweeted a poll asking a simple question:

If there was an open, semi-formal journal, would you submit your papers, talks, research for publication?

This poll was seen 14,862 times, received 55 retweets, and numerous replies; there were 204 votes, which break down like this:

  • Yes: 42%
  • Maybe: 27%
  • No: 7%
  • Why is this needed? 24%

The last number is the most interesting to me: to many of us, the issues are clear and of increasing importance, to others, it’s less so. When I posted this poll, I knew that number would be interesting, but at 24%, it’s more significant than I expected. There are, of course, academic journals available, though they are not suited to the needs of the community – nor entirely appropriate for the research that is published. This shows the deep cultural gap between academic and practitioners, and why purely academic journals haven’t been able to address these needs.

In the replies, a number of questions were raised, which reveal some interesting issues and concerns:

  • I don’t write formal papers, I don’t know what would be expected.
  • I can present at a conference, but formal papers make me uncomfortable.
  • Do I have to pay to have the work published?
  • How would this be funded?

There are a number of interesting things here, the most significant I see is that it’s clear that the publishing model used for academic journals doesn’t work for the community. This is often independent work that has little to no funding, so there are no grants, no assistants to help with the paper, and no familiarity with the somewhat unique world of academic journals.

For such an effort to succeed, a number of objectives would have to be met:

  • No cost to publish.
  • No cost to access.
  • Simple submission requirements to minimize pain and learning curve.
  • Cooperation with conferences to encourage submissions.
  • Regular publication schedule.
  • Volunteers willing to coordinate and edit.
  • Community control, no corporate interference or control.
  • All rights should remain with the authors, with license granted for publishing.
  • 100% non-profit.

In my view, a new organization needs to be created, with the goal of becoming an IRS recognized non-profit, with a board of directors elected by the community to guide the creation and publication of this journal. Funding should be from organization members and corporate sponsors; with a strong editorial independence policy to ensure that sources of funding can not interfere with their decisions or what is published.

The journal should strive for sufficient peer review and editorial quality to ensure that the journal is recognized as a legitimate source of trustworthy information, and as a permanent archive of the collected knowledge of the industry. Access to this information should be free to all, so that knowledge spreads, and is not locked behind a paywall or left to perish – unknown and unseen. The journal should strive to be as complete as possible, working with researchers, with companies, and with conferences to collect and published as much high quality research as possible.

Publication in this journal should be a matter of pride for authors, something they advertise as an achievement.

Path Forward #

To move forward with this, will require the help and support of many people – it is not a simple task, and comes with many complications to succeed. Though as the industry and community grow, it’s clear to many that a solution is needed for this problem. The knowledge produced needs to be collected, and made easy to find, easy to cite, and freely available to all of those that seek it.

Adam Caudill


Related Posts

  • Proposal: Association of Security Researchers

    Security researchers play an important role in the industry, though one that doesn’t always receive the support needed. In this post, I am proposing the creation of a new non-profit entity, the International Association of Information Security Research Professionals (IAISRP), as a supporting group to push research forward, and provide the tools and resources to improve the quality of work, and the quality of life for those involved in this vital work.

  • The Manifesto

    As a child, all of my time was spent reading – at the age of 8 or 9 I was staying up all night reading the likes of Dickens and Verne, at 11 or 12, I was tearing through encyclopedias, medical texts, and anything else I could get my hands on. I had a love for learning, for understanding, a desire to know everything, and an insatiable curiosity that often led me in interesting directions (in that ancient curse “may you have an interesting life” kind of way).

  • Making BSides Knoxville

    Two years of discussions, months of planning, weekly meetings, and thousands of dollars – BSides Knoxville 2015, the first BSides Knoxville that is, is in the books. By any metric I can think of, it was a resounding success – the feedback was great, awesome talks, good food, and a great atmosphere. I would like to give a little insight into the event, some of what I learned from it, what went right, went wrong, and how to make something like this without going insane.

  • On Apple, Privacy, and Device Control

    If you’ve bothered to look at Twitter or any technology news source, you’ve seen that Apple made a major announcement: Expanded Protections for Children. This has been written about by countless outlets, so I’ll assume you’re familiar with the basics. The announcement covered a few new features being added to the next version of Apple’s operating systems, namely: Scanning of inbound and outbound messages for sexually explicit images. Scanning images being uploaded to iCloud for CSAM.

  • Utilitarian Nightmare: Offensive Security Tools

    Or: Ethical Decision Making for Security Researchers. There has been much discussion recently on the appropriateness of releasing offensive security tools to the world – while this storm has largely come and gone on Twitter, it’s something I still find myself thinking about. It boils down to a simple question, is it ethical to release tools that make it easy for attackers to leverage vulnerabilities that they wouldn’t otherwise be able to?