FAQ

This page will be updated with questions (and answers) as they arise. Please check back regularly. If you do not find an answer to your question here, you are welcome to contact the IJCAI-22 workflow or program chair at workflow@ijcai-22.org or pcchair@ijcai-22.org, but please make sure that you have read IJCAI-ECAI 2022 CFP and the current FAQ first.

Please do watch our videos on “Reviewing” and on the “Review process explained” before starting to review. The REVIEW form, the  guidelines for reproducibility, and the ethics policy are also important. 

Attendance

 

For large fragments of the IJCAI community physical travel is now possible, and the pandemic-associated risks – greatly reduced by vaccination – have ceased to generally outweigh the advantages of the in-person interaction. Therefore, the conference committee choose to make IJCAI-ECAI an in-person conference. However, not everyone is able to attend. Therefore, IJCAI-ECAI-2022 has a limited virtual component to enable the participation of major sponsors and of delegates who cannot travel pursuant to the Covid-19 regulations of their country of residence.
The in-person component of IJCAI-ECAI 2022 will operate as follows:
  • it will consist of in-person presentations; talks at IJCAI-ECAI 2022 will not be live streamed because the quality of the live-streaming cannot be guaranteed and because hybrid sessions are rarely a good experience;
  • the papers accepted to the main tracks of IJCAI, AI for Good, AI – Arts & Creativity, Survey, Demo, Journal, and Sister Best Paper tracks can be divided into two subsets depending on whether they will be presented in person in Vienna, or not;
  • for the papers in these tracks to be presented in person in Vienna, there is both a talk and a poster;
  • the duration of the talks in Vienna is the same as in the video recorded for SlidesLive (6, 12 or 15 min) and there will be no Q&A at the end of these talks as there is an accompanying poster for these papers where discussions can take place;
  • the talks scheduled on Tuesday or Wednesday have posters on Tuesday, those scheduled on Thursday or Friday will have posters on Thursday; we recommend EU-size A1: 841 x 594 mm (23.4 x 33.1 in) or US-size (24 x 36 in) in portrait orientation for posters;
  • the papers in the Early Career Track are handled in the same way as the other main tracks, except that there is no associated poster;
  • the schedule for all in person talks is available at https://www.ijcai-22.org/ijcai-2022-schedule/

 

The virtual component of IJCAI-ECAI 2022 will operate as follows:
  • it will operate through the Whova app and will be available to all registered participants;
  • there will be access to all pre-recorded video’s for the presentations in the main tracks of IJCAI, AI for Good, AI – Arts & Creativity, Survey, Demo, Journal, Early Career and Sister Best Paper track;
  • there will be neither online poster sessions nor online presentations;
  • the papers that are not presented in the in-person conference will be clustered into thematic topics; for each topic, there will be a suggested meeting time where registered authors and other registered persons can discuss the papers and interact with the authors; the suggested meetings will be scheduled at 10 am, resp. 11 am (CET, Vienna), which corresponds to 4 pm, resp. 5 pm in (CST, Beijing); the schedule for these meetings will become available in the next week;
  • as there will be no online presentations attendees are assumed to watch the video’s of interest by themselves;
  • Whova provides for each thematic topic also a Q&A forum, where authors and participants can interact via messages, and new thematic topics and meetings can be created by participants.

For workshop, doctoral consortium and diversity and inclusion talks and events, no support for interaction or access to pre-recorded video’s is foreseen. In case (registered) authors are unable to present, it is at the sole discretion of the event organisers to give them the opportunity to pre-record a short video and to project it during their event. All registered participants can communicate via the Whova app.

Discussion

If the paper is a resubmission, we should have three reviews and take into account the resubmission file (containing the reviews of the previous submission and a list of changes made). In all other case, we should go for (at least) four reviews.

Please start by reading the other reviews, and the author rebuttal and resubmission files (if available).
On the basis of this information, you may consider revising your opinion and review. You may also disagree with
the other reviewers. In both cases, please discuss with your colleagues in order to arrive at a consensus decision.
At the end of the process, please revise your review to reflect possible new insights and answer the last question.

According to the instructions, the reviewers should not have asked new results and the authors should not have provided them.
It is the current version of the paper that is under review, not possible future versions.
New results should in any case be ignored.  Furthermore, if no reviewer asked for new results in the “pressing question” section, the whole rebuttal should be ignored.

Please take into account only 3/4 of the rebuttal, and ignore the last 1/4.

If the rebuttal is very short (that is, it is obvious that the rebuttal would occupy less than 1 column in the right template),
you may take it into account. Otherwise, please ignore it.

You can find that by clicking on the paper ID and looking for the resubmission file.

It most likely means that the paper has been withdrawn by the authors since the rebuttal. In any case, no further action is needed on such papers.

They can send the email via the “Email Senior meta-reviewer” button on any of their assigned papers. Area Chairs are assigned SPCs so the papers of an SPC all have the same Area Chair and it does not matter via which paper the SPC contacts them for the ranked list. 

Rebuttal

The author response is intended to provide answers to pressing questions by the reviewers, to point out factual errors in the reviews, and to appeal to unethical reviews. The author rebuttal is not intended to start a dialogue between the authors and the reviewers.

Please do not respond just to say that you don’t like something in the reviews. The cases where you can respond are the following:

1- A reviewer explicitly asks “pressing questions” in part 4 of his review.  Typical questions include requests to clarify or justify particular issues, or about important relationships to other works.

2- You have detected a factual error that could be used to recommend rejection. 

3 – In case of unethical reviews, you can enter a confidential message, which will only be visible to the Senior PC Members, Area Chairs, and (Associate) PC Chairs.

Before investing a lot of time in the rebuttal, please ask yourself also the question whether your answers have a chance to tip the balance. For instance, if there are four reviews that all recommend some form of reject, it is unlikely that the rebuttal will turn this paper into an accept.

 

If you are in one of the cases described in the last question FAQ.{When can I write a response?}, then you can write a response. For case (1), please provide the information that the reviewer requested, and nothing else. For case (2), you have to clearly describe what is erroneous in the review and why. Your response to (1) and (2) is limited to 1-page, using the template provided by IJCAI-ECAI 2022 at www.ijcai.org/sites/default/files/Guidelines/ijcai22-authors-response.zip.

Your response to (3) can be entered in the separate confidential box. 

Please notice that less is more, and that shorter and more crisp responses have a higher chance of convincing the reviewers.

 

The PC members, Senior PC Member, Area and Program Chairs involved in judging your paper can see the pdf file, i.e. your reply with regard to factual errors and responses to questions of the reviewers. Your confidential remarks are only visible to the Senior PC Member, Area Chairs and (Associate) PC Chairs, not to the PC members. 

No. Before investing a lot of time in the rebuttal, please ask yourself also the question whether your answers have a chance to tip the balance. For instance, if there are four reviews that all recommend some form of reject, it is unlikely that the rebuttal will turn this paper into an accept.

Every paper will get at least three reviews. For the vast majority of papers, there will be four or more at the end of the reviewing process. For some resubmissions, we target three reviews.  At the time the rebuttal opens, some reviews were late, but please observe that they can still come in during the rebuttal phase, and also that during the discussion phase further reviews may be added. 

Yes they can. Despite our best effort to get all reviews in time, some reviewers are late, and we are really sorry about that.

You may view a review as unacceptable when it is overly thin and uninformative.  Senior PC members and Area Chairs are monitoring and also rating the reviews. In case reviews are overly thin and uninformative (as judged by the SPCs and ACs), they will either be expanded, or not be taken into account. Other reviews may be added in such cases.

You may view a review as unacceptable because it is unethical. These are the two cases where you can use the confidential box in the rebuttal.

All authors can submit or update the response. It can be updated as many times as you want before the response deadline.

Go to the column “Action” in your author’s page, click on “Post Author Feedback” and follow the guidelines.

No, you cannot. This is providing extra results or materials w.r.t. your original submission.
At the same time, you can point out that — as sketched in our reproducibility guidelines — submitting code is not a requirement. Please point the reviewer to these guidelines

No, you cannot. New experiments are new results, they cannot be included.
Furthermore, reviewers should not ask for this as is clear from question 4 in our review form.
There it is explicitly stated that questions should not ask for new results.

Yes, you can, as this is not a new result, it is an explanation and clarification of what is in the paper.

No, you cannot. The reason is that we want these guidelines to be clear to both authors and PC members as to avoid misunderstandings.

You can withdraw you paper before February 15. If you withdraw your paper after that date, it will automatically be a rejected paper (which you may have to declare when submitting to further conferences).  If you want to do this, please contact .

Not much. IJCAI-ECAI 2022 has done major efforts and sent numerous reminders to reviewers to remind of the deadlines. While the vast majority of reviews has been received and further ones are coming in,  some reviews will not arrive in time for the rebuttal.  Further reviews have been or will be solicited and at the decision time, there should be enough information available to make an informed decision. At this stage,  we can only ask all reviewers with still open reviews to complete them as soon as possible.

 

Reviewing

Please do watch our videos on “Reviewing” and on the “Review process explained” before starting to review. The review form, the guidelines for reproducibility, and the ethics policy are also important.

The reviewing process consists of two phases. In Phase 1, every paper will be reviewed by two reviewers and if a paper receives two reviews that are not sufficiently positive, it will be rejected without any opportunity to submit an author response.  By submitting a paper, authors acknowledge that they are aware of the possibility of receiving a summary rejection notification.

Authors of summary rejected papers will receive two reviews.

The IJCAI series of conferences started in 1969, and since then IJCAI has remained the premier conference bringing together the international AI community. In classic AI areas, with a long tradition at IJCAI, people submit their best papers to IJCAI.  For areas that have grown a lot in the last few years (NLP, vision, machine learning), a rule of thumb to decide whether a paper is good enough for IJCAI, is to ask whether it is good enough for conferences such as ACL / ICCV / ICML.

The purpose of the rebuttal is to ask questions about specific issues that 1) could directly influence your evaluation of the paper, and 2) do not require providing new results. Typical questions include requests to clarify or justify particular issues, or about important relationships to other works.

The purpose of the rebuttal phase is not to enter a discussion about the ways the authors could extend or revise their paper by, for instance, asking for extra experiments or theoretical results. You must judge the current and not the possible future versions of the paper. 

The rebuttal will be visible to all members of the program committee involved in the paper (reviewers, meta-reviewers, senior meta reviewers, (associate) pc chairs).

There is also the option to enter confidential comments that will not be visible to the reviewers, but will be accessible to the meta-reviewers, senior meta reviewers, and (associate) pc chairs.

All papers violating the rules specified in the CFP must be rejected. Most papers that obviously violated these requirements have been desk rejected before the reviewing phases. However, in case you find an obvious violation, please inform the workflow chairs (at workflow@ijcai-22.org) immediately. You can also mention it in your review, though you must not take it into account in your judgement as decisions about violations are made by the PC-chair directly in order to ensure consistency and fairness.

Please refrain from using search engines such as Google to try to identify the authors.  This is not good practice.

Please start by watching our video  “Reviewing” and looking into our review form.

The program chair highly recommends reading Mistakes reviewers make (by professor Niklas Elmqvist), see https://sites.umiacs.umd.edu/elm/2016/02/01/mistakes-reviewers-make/

It provides a very balanced view on reviewing that applies to AI and its subfields. (One little detail: when Elmqvist writes about user study (coming from an HCI perspective), you might want to think of experiment (in AI or ML).)

The program chair also highly recommends going through the slides that Hendrik Blockeel and Jesse Davis used for a tutorial on reviewing, and that they have kindly adapted   for IJCAI-ECAI 2022.

https://dtai.cs.kuleuven.be/introduction-to-reviewing-for-ijcai.pdf

The assignment algorithm considers your bids, the subject areas, TPMS, as well as the reviewer workload, etc. It aims at optimizing the global assignment. If you did not bid, or entered positive bids for too few papers, or mainly on very popular papers, the algorithm may not have found the best papers for you. Still there are many other papers that need to be reviewed, and we rely on your expertise to provide an opinion.

Warning: If you see that a paper does not at all match the subject areas entered for that paper, please inform  workflow@ijcai-22.org and put it in your review, as this is considered bad practice. 

If a paper is somewhat outside your core area of expertise, your evaluation of the work as an informed outsider can still be useful. Nevertheless, in exceptional cases where you believe you are unable to provide any well-informed review, please inform workflow@ijcai-22.org as soon as possible. We will then try to assign it to someone else. Obviously, the longer you wait, the more difficult it is to reassign. (The same holds if you discover a conflict of interest, or if you cannot review a paper for any other reason.)

In principle, the answer is NO, unless there is really no other option because of unforeseen circumstances (like illness). The reason is that for large conferences like IJCAI, it is extremely hard to control the conflicts of interest if subreviewers are allowed. In any case it is always the PC member / reviewer  that remains responsible for the review.

Please continue to review it. We don’t regard this situation as a COI. Reviewers for previous versions of the submission are in an ideal position to judge whether the current submission has been really improved. Note that submissions that have been rejected at any peer-reviewed conference within the past 6 months must declare the resubmission and attach the previous review as well as an explanation about how authors have improved the work. The resubmission file will be visible only after the rebuttal during the discussion period.

As mentioned in the call for papers, IJCAI-ECAI 2022 welcomes papers on AI techniques for novel application domains. Keep in mind though that IJCAI is the place to publish research papers.  Therefore, if the use of AI, or the AI technique used, is novel for this kind of application, it can be considered, provided that the application is formalized in such a manner that the formalization can be applied to solve other applications in an innovative manner as well.”  If the paper doesn’t completely satisfy the previous criterion but it clearly explains why the used technique works well, again it can be considered. In those cases, acceptance depends on how important the application is, how significant the improvement is, and to which extent it can be generalized to other problems. If the paper doesn’t satisfy the previous criteria but the AI technique used has a major impact in the application domain and in industry or society, it may be considered for publication at IJCAI-ECAI 2022.

On the other hand, simply applying a standard AI or ML technique (like neural nets or decision trees) on standard datasets from an application domain clearly does not constitute a valid contribution for IJCAI-ECAI 2022.

That is not a problem as long as the papers  are presented in non-archival venues (that is, as long as there are not formally published proceedings with a publisher, in general with a DOI, ISBN, or ISSN ).  ArXiv submissions are also ok. 

No, if the short paper is not longer than a two-column two-page paper. If the ‘short’ paper is longer than that, but no longer than half of the IJCAI paper, then the extension must be significant enough.

The call for papers mentions: 

“For anonymity of the IJCAI-ECAI 22 submission, in the citation please hide the authors’ names, and hide the place where the abstract got published. However, you need to mention that this is an extension of a published abstract, and submit the published abstract as supplementary materials (with authors’ names/affiliations and publication venue removed). For example, you should mention:

This submission is an extension of a published extended 2-page abstract [AAA].

Ref-AAA: XXX. Short paper title. YYY, 2020. [see: supplementary file]”

Submissions

7-10-14 January are the deadlines for the MAIN track. The special tracks such as tutorials, surveys, AI for Good,  AI, Arts and Creativity, have later deadline. For many of these, the submission is not yet open.

Yes. This applies more generally to papers presented in any non-archival venue (that is, as long as there are no formally published proceedings with a publisher, in general with a DOI, ISBN, or ISSN ). 

​​IJCAI requires that all submissions be anonymized. To facilitate this, we require that the authors follow both the letter and spirit of the normal anonymization guidelines. Please ensure also that the PDF meta-data does not reveal the authors’ identities.  Any paper that obviously violates this requirement will be rejected without review.

Yes, you can but only if you 1) retract your paper in case it is accepted, and 2) include the necessary information (reviews and response letter) in case it was rejected. This must be done by the full paper submission deadline (January 14).

Yes. Resubmissions from any peer-reviewed conference within the past 6 months have to follow the “Resubmissions” terms in the call for papers, otherwise the paper may be rejected without further review.

 

Please make sure you have read the “Resubmission” part of CFP to know the details about the resubmission.

To submit a resubmission file PDF, first upload your submission. You will then be able to upload the resubmission file from the author console as a supplementary file. There are two entries for supplementary files: one is “TechnicalAppendix”, which is the normal supplementary file for the main paper; and the other one is “ResubmissionFile”, to which you should upload the resubmission file. Note that the maximal size of the resubmission PDF is 10M.

No file can be submitted after the full-paper submission deadline.

Your resubmission information will be visible only to reviewers after they enter their review, and after the rebuttal. 

A paper is considered a ‘student paper’, and thus a potential winner of the ‘Best Student Paper Award’, when the first author is a student who, at the time of submission, has not obtained any PhD degree yet. When submitting a paper, this should be indicated by marking the appropriate checkbox.

Yes. If, for instance, the short paper is a two-column two-page paper. If the ‘short’ paper is longer than that, but no longer than half of the IJCAI paper, then the extension should be significant enough. Significance will be assessed by reviewers. In any case, the short version has to be cited and it must be clear that the IJCAI submission is an extension.

For anonymity of the IJCAI-ECAI 22 submission, in the citation please hide the authors’ names, and hide the place where the abstract got published. However, you need to mention that this is an extension of a published abstract, and submit the published abstract as supplementary file (with authors’ names/affiliations and publication venue removed). For example, you should mention:

This submission is an extension of a published extended 2-page abstract [AAA].

Ref-AAA: XXX. Short paper title. YYY, 2020. [see: supplementary file]

Title: It is highly recommended not to go above two lines in the IJCAI format. Three lines is a strict maximum.

Abstract: There is no word limit for abstracts. However, 200 words is a nice upper limit that authors should always apply.

By ‘abstract submission’ we mean that you should register your paper by giving a title and a short abstract. This abstract is intended for helping program committee members get papers that match their expertise. There are neither abstract-only submissions, nor any paper-only submissions: in all cases, you must submit an abstract, and then you must submit a full paper. (See abstract submission and paper submission deadlines on the web site).

You can always include references in the six main body pages, in addition to the last reference page.

For references and tables, \small is OK, but no smaller than \small, otherwise, the submission may be rejected without review.

Please ask your co-authors to log in to the submission system and fill in author information. Please make sure that you use the email address they prefer to use in CMT.   As an author or co-author, once you log in, you can see all papers under your name or account  (if you have multiple accounts/email addresses, you should check all your accounts). If an author finds any submission, for which they did not give their consent to submit, they should report to the program chair immediately.

Please use GoogleScholar page or ResearchGate page. If you do not have any publications yet, you can leave this information empty.

Authors are also required not to submit their papers to venues with formally published proceedings during the  IJCAI-ECAI 22 review period. These restrictions do not apply to workshops and similar specialized presentations with a limited audience and without published proceedings.

You can withdraw you paper before February 15. If you withdraw you paper after that date, it will automatically be a rejected paper
(which you may have to declare when submitting to further conferences).

Every so many years, IJCAI is organised together with either PRICAI or ECAI, and then the conference becomes a joint IJCAI-ECAI or IJCAI-PRICAI.
The conference is then organised under the rules of IJCAI. There is only one submission site, one proceedings, and one conference then.

Other Tracks

The deadlines for the different special tracks differ, and also all special tracks have separate submission sites. Some of them like the AI for Good, and the AI, the Arts and Creativity track also accept different types of submissions. Please check the details of the respective calls.

This depends again on the track and the kind of submission. Papers accepted to the following tracks will appear in the main IJCAI proceedings: Survey, AI for Good, Demo and AI, Arts & Creativity.

Program Committee

The Program Committee for IJCAI-ECAI 22 consists of:

– Area Chairs, who overview the activities of the Senior PC members and resolve borderline cases;

– Senior PC members, who draft the meta-reviews, and guide the discussion amongst the PC members;

– PC members, who write reviews and participate in the discussion phase.

Notice that there is one layer less than for IJCAI 21.

IJCAI-ECAI 2022 reviewers produce a small number of reviews (typically 2),
where IJCAI-ECAI 2022 PC-members are expected to typically review 4-6 papers.

Triple blind reviewing: From the perspective of the Senior Program Committee members (SPC) and Program Committee members (PC),  the reviewing process will be triple blind, i.e., SPC/PCs cannot see the identities of authors and other SPC/PCs, and vice versa. Author identities are also invisible to Area Chairs (ACs) and vice versa.

The reviewing process proceeds in two phases. In Phase 1, every paper will be reviewed by two reviewers and if a paper receives two reviews that are not sufficiently positive in Phase 1, it will be rejected without any opportunity to submit an author response.  By submitting a paper, authors acknowledge that they are aware of the possibility of receiving a summary rejection notification.

Authors of summary rejected papers will receive two reviews.

The IJCAI series of conferences started in 1969, and since then  IJCAI has remained the premier conference bringing together the international AI community. In classic AI areas, with a long tradition at IJCAI, people submit their best papers to IJCAI.  For areas that have grown a lot in the last few years (NLP, vision, machine learning), a rule of thumb to decide whether a paper is good enough for IJCAI, is to ask whether it is good enough for conferences such as ACL / ICCV / ICML.

All papers violating the rules specified in the CFP must be rejected.