, University of Arizona and Chinese Academy of Sciences
Pages: pp. 2-4
Abstract—What is it, exactly, that technical magazines want? Here, IEEE Intelligent Systems' Editor in Chief discusses common misconceptions and tips for potential contribution authors, to help them publish their work.
Keywords—AI communities; peer review; intelligent systems
In my first letter as Editor in Chief, published in the January/February 2013 issue of IEEE Intelligent Systems, I discussed three short-term tasks for our publication. The first is concerned with developing a strategy to communicate with the potential author base of precisely the kinds of articles that IS wants. The second targets strengthening connections with mainstream AI communities. The third is about implementing a social media strategy to broaden the reach of IS.
We're making steady progress on all three fronts. As you'll see in a bit, this article is part of the effort related to the first task.
On the second task, we’ve taken multiple approaches that yielded good results. Many senior AI researchers kindly helped us generate an impressive list of candidates for the 2013 “AI's 10 to Watch” awards. The final winning group has been truly outstanding, with high-impact, noteworthy research contributions. This year, the “AI's 10 to Watch” list was widely distributed at the AAAI and IJCAI meetings, inspiring young AI students and researchers and the community at large. In addition, IS has committed to publishing one “core” AI issue per year, following the advice of the IS advisory board—in particular, Austin Tate, Bill Swartout, and Jim Hendler. And one editorial board member, Ulrich Furbach, has been working hard on the first “core” AI issue on representation and reasoning, which is scheduled to appear in the first issue of 2014.
On the third task, IS‘s lead editor, Brian Kirk, has invested a lot of energy in developing a comprehensive social media strategy for IS and started the implementation. The magazine now has a home in all major social media outlets and is building up its circles of “friends” and “followers.” We'll report on our experience early next year.
This letter is part of our coordinated effort to communicate with potential contributing authors regarding the mission of IS and the kinds of articles IS wants. In what follows, I'll informally discuss the magazine's scope and intended audience. I'll also try to clear up some misconceptions and touch upon the primary reasons that articles get rejected.
As the next step of our communication campaign, in the next month or so we'll be sending out a formal general call for papers to various outlets to encourage high-quality submissions.
Note that the following discussion is neither comprehensive nor systematic. It's meant to provide editors’ perspectives (in a collective sense) as manuscripts go through the review process. The authors who plan to contribute to IS are encouraged to visit www.computer.org/portal/web/computingnow/intelligentsystems—especially review the “About IS” section and the posted author guidelines for a fuller exposition.
IS is a premier technical publication on AI theory, methods, and applications. Its intended audience includes researchers and professionals from a variety of fields who develop, apply, or benefit from AI frameworks and techniques. Given the prevalence of AI applications, these fields almost encompass the entire spectrum of science, engineering, medicine, business, military, and government. With the exception of various departments and columns, all IS content is peer reviewed.
For a submission to sail successfully through the peer-review process, the first necessary condition it must satisfy is that the topics covered are relevant to the magazine. In the context of IS, relevance has two aspects. First, the submission must have strong relevance to AI. The editorial board and the reviewers take a very broad, inclusive perspective as to what AI is. The list of topics published on the IS website reflects this perspective.
Second, the submission needs to establish some relevance to some applications. Purely theoretical treatments perhaps are more suited for other publications. However, this practical, application-oriented relevance doesn't necessarily mean that a system has been developed and deployed in real-world settings. In many cases, potential practical applicability is sufficient so long as the application contexts and potential impact are clearly presented.
IS publishes novel technical materials. Novelty can come in many different forms and isn't exhaustible. Technical innovations could be presented by using either a methodological or application perspective. Contributions could be shown as new advances in tools, specific techniques, algorithms, application contexts, conceptual frameworks, and ways of formulating or representing problems, among others.
In at least one aspect, a successful submission should clearly demonstrate that novel contributions have been made, with potential or realized real-world impact. Submissions examining unconventional, cutting-edge thinking or practice have a much better chance than those focusing on incremental advances following well- established frameworks or techniques.
Work publishable at IS needs to be significant on a technical level, given the publication's very low acceptance rate. Evaluating such significance can be tricky and is often subjective.
One recommendation for the submitting authors is to carefully examine the “Who cares?” question. Is a well-known open research problem being solved? Will peers working in the same field care about the problem being addressed or the solution proposed? Will fellow researchers working in related fields benefit from the findings? When applied, will the proposed technical work deliver major value? Can the ideas or techniques be adapted and used in other application contexts?
How to measure technical quality can be highly discipline-specific. Given AI's broad scope, authors must be mindful about the commonly accepted evaluation framework in their fields. We're certainly not proposing some kind of rigid, template-driven research. But in general, a good paper should clearly and cleanly discuss the state of the art, identify the specific research challenges or gaps, and articulate exactly what's new about the proposed approach.
Whenever possible, the proposed approach should be formalized in the proper theoretical or computational context. When appropriate, concrete evidence supporting the technical claims about the work presented needs to be shown. Such evidence could be formal properties, results of algorithmic analysis, computational evaluation, prototyping, user studies, or convincing case scenarios.
At the risk of repeating myself in such a short letter, here are several common scenarios that result in papers being rejected.
Because some authors misunderstand that IS is a technical publication, they submit non-technical contributions such as position papers or industry recommendations. Sometimes IS is still interested in such contributions, provided that they're cutting-edge and provide thought-provoking discussions. However, the “regular manuscript” category isn't the proper channel in such cases. Authors are encouraged to contact the magazine's department editors directly to find out whether such contributions could be reviewed and used as department articles.
It's important to clear up this misconception, because this isn't how these publications work. Due to page limits and a magazine's intended crisp presentation style, it's true that IS probably won't publish full technical details, such as an exhaustive listing of properties or detailed proofs. However, regarding the level of novelty, significance, and technical quality present in this magazine, IS is as demanding as the best journals.
This happens frequently, where the solution framework or the underlying formalism is still a work in progress, and the evaluation is lacking. The chances for such papers to survive the review process are low. Some authors mistakenly think that shorter papers mean lesser contributions and submit their preliminary work to publications with a lower page limit. To remedy this situation, a good rule of thumb is this: if authors don't think that their work will have a chance at the best technical conferences in their field, it's likely that such papers won't fare better in the peer review process of IS.
The purpose of this article is to encourage high-quality submissions to IS, and clear up some misconceptions. We hope that by sharing the editors’ perspectives, contributing authors, especially our junior AI colleagues, will be able to better target their work for suitable outlets, improve how they conceptualize and detail their work for publication, and increase their chance of getting manuscripts accepted. Happy researching and paper writing!