LitFetchR (Tutorial)

Setup

We strongly recommend creating a new R directory for each individual review project. To do so in RStudio:

• Go to File>New Project…>New Directory>New Project

• Choose a name for your directory (e.g., review_fish_vibrio), then Browse... to choose where to save the directory.

  We suggest to save the directory on your computer (e.g. C:/users/...) instead of clouds (e.g. OneDrive, Box, etc.), as restriction access from institution might block R.

• Click Create Project

From your new directory, install LitFetchR from CRAN:

#This step can be skipped if `LitFetchR` is already installed
install.packages("LitFetchR")

Alternatively, install the development version from GitHub:

#Install using devtools package
install.packages("devtools")
devtools::install_github("thomasdumond/LitFetchR")

#or

#Install using remotes package
install.packages("remotes")
remotes::install_github("thomasdumond/LitFetchR")

Two optional packages are only needed for some functions:

• Deduplication (dedup = TRUE) uses ASySD, which is not on CRAN. Without it, the retrieval works with dedup = FALSE:

install.packages("remotes")
remotes::install_github("camaradesuk/ASySD")

• Scheduling on macOS and Linux uses cronR: install.packages("cronR"). On Windows, the Windows Task Scheduler is used and nothing more is needed.

Then load the package:

#Load `LitFetchR`
library(LitFetchR)

Save API keys

Save your API keys in your personal R environment:

#This step can be skipped if your API keys have already been saved. Repeat this step if your API keys changed.

#Load `LitFetchR`
library(LitFetchR)

#Save API keys in your R environment

#Example of what you should see:
#> save_api_keys(wos_api_key = "abcd01234", scp_api_key = "efgh5678")
#Saved key(s) wos_api_key, scp_api_key to -path-to-your-renvironment/.Renviron.
#Restart R for the new environment variable(s) to be available.

This will allow LitFetchR to locally access your personal API keys while keeping them confidential if you need to share your code with collaborators.

save_api_keys() accepts the following, in any combination:

• wos_api_key — Web of Science API key.
• scp_api_key — Scopus API key.
• scp_insttoken — optional Scopus institutional token, required by some institutional subscriptions.
• ncbi_api_key — optional NCBI/PubMed key, for a higher PubMed request rate.

For instructions on how to obtain each of these, see Get API keys. The optional tokens can be saved at the same time as your keys, for example:

save_api_keys(scp_api_key = "efgh5678",
              scp_insttoken = "your-insttoken",
              ncbi_api_key = "your-ncbi-key")

As mentioned by R, it is important to restart your session before continuing:

Session>Restart R

Then load LitFetchR again using library(LitFetchR)

If you don’t have API keys for Scopus and Web of Science, see Get API keys.

Save the review search string(s)

The interactive create_save_search() function will guide you through its workflow. Here is an example of what you should see:

#Load LitFetchR
library(LitFetchR)

#The following function can help to create your search string(s)
#It has to be used to save your search string(s), we recommend choosing the R project directory as the directory in the function.

# Example of what you should see building the search: fish AND "vibrio harveyi" AND diagnostic
#
# > create_save_search(wos = TRUE,
#                      scp = TRUE,
#                      pmd = TRUE,
#                      directory = "C:/path-to-project-directory")
#
# History had been created.
# Enter your search string (or "summary" or "exit"): fish
# [1] "fish"
# [1] "Web of Science: 1793296 results"
# [1] "Scopus: 718644 results"
# [1] "PubMed: 384742 results"
# Enter your search string (or "summary" or "exit"): fish AND "vibrio harveyi"
# [1] "fish AND \"vibrio harveyi\""
# [1] "Web of Science: 2084 results"
# [1] "Scopus: 1080 results"
# [1] "PubMed: 727 results"
# Enter your search string (or "summary" or "exit"): fish AND "vibrio harveyi" AND diagnostic
# [1] "fish AND \"vibrio harveyi\" AND diagnostic"
# [1] "Web of Science: 126 results"
# [1] "Scopus: 22 results"
# [1] "PubMed: 106 results"
# Enter your search string (or "summary" or "exit"): summary
#                                Search_Term Results_WOS Results_SCP Results_PMD
# 1                                     fish     1793296      718644      384742
# 2                fish AND "vibrio harveyi"        2084        1080         727
# 3 fish AND "vibrio harveyi" AND diagnostic         126          22         106
# Do you want to save a search string for automated retrieval? (yes/no, Enter to cancel): yes
# Select the index number of the search string to save (1 to 3, Enter to cancel): 3
#
# Selected search string: fish AND "vibrio harveyi" AND diagnostic
#
# Enter a name for the search identification (Enter to cancel): fish_vibrio
# Search string saved successfully.
# Enter your search string (or "summary" or "exit"): exit
# Exiting search tool.

Typing exit before any search string was saved offers to save one, so a search is not lost by mistake. Every question can be cancelled by pressing Enter, and an invalid answer is asked again.

Exclusions: type NOT or AND NOT, LitFetchR writes the exclusion the way each platform requires (AND NOT for Scopus and NOT for PubMed and Web of Science). Phrases between double quotes are never changed.

After using create_save_search(), two new files will appear in your directory:

• search_list.txt

  This file contains the name and search string that you saved using create_save_search(), one name=search string per line. It is an essential file to retrieve the reference later. If you modify the search string in this file, the modified search string will be used for future reference retrieval. The file is checked before each retrieval and the run stops with a clear message if it is missing, empty or malformed (for example a line without “=” or a name used twice). The search string actually used is written to “run_log.csv” at each run.

• history_search.xlsx

  This file contains all the searches that you ran when using the function create_save_search(), with the search string sent to the literature platforms, the number of results that they returned and the time of the search. Each run of the function adds rows to the same sheet.

If you already validated your review search string, use create_save_search() to save the search string and setup the file required to continue toward the automation of reference retrieval.

Setup the automatic reference retrieval

The automated reference retrieval uses either Windows Task Scheduler or Cron (Mac/Linux). You do not have to specify it, LitFetchR will detect the system and setup the task for you using the following. On macOS and Linux the package cronR must be installed.

#Load LitFetchR
library(LitFetchR)
#We recommend choosing the R project directory as the directory in the function. It has to be the directory where "search_list.txt" and "history_search.xlsx" have been saved.
#We recommend using a single word for the *task_id* or to use underscores "_" to separate words.
#The retrieval frequency is currently available "DAILY", "WEEKLY" or "MONTHLY".
#You need to use a 24H format for the time of reference retrieval.
#For "WEEKLY", `day` is the weekday (MON, TUE, WED, THU, FRI, SAT or SUN).
#For "MONTHLY", `day` is the day of the month (1 to 28).
#If you do not give `day`, the weekday (or day of the month) of today is used.
#If you do not have an API key for WOS and/or SCP or want to exclude any database,
#change "TRUE" to "FALSE" in front of the corresponding database (e.g.`wos = FALSE`).

#Example of what you should see
#> auto_LitFetchR_setup(task_id = "fish_vibrio",
                     # when = "WEEKLY",
                     # time = "14:00",
                     # day = "MON",
                     # wos = TRUE,
                     # scp = TRUE,
                     # pmd = TRUE,
                     # directory = "C:/path-to-project-directory",
                     # dedup = FALSE,
                     # open_file = TRUE)
#To get a RIS file instead of a CSV file (for example for Covidence), add `output_format = "ris"`.
#Task scheduled!
#  Task name:  fish_vibrio
#  Recurrence: weekly on Monday at 14:00
#  Next run:   (date and time of the next run)
#  Status:     Ready

Task scheduled! is only shown after the scheduler accepted the task and the task was read back from it. Check the summary that follows: it is your proof that the retrieval will run. If the scheduler refuses the task, the function stops and shows the message of the scheduler. If the task cannot be confirmed, a warning is shown instead. A task with the same name is never overwritten, remove it first with remove_scheduled_task().

After running auto_LitFetchR_setup(), one new file is created in your directory:

• auto_LitFetchR_code_READ_ONLY.R

  This file contains the code that is automatically run by the scheduled task (uses the function manual_fetch()). It is set as “READ ONLY” to avoid accidental modification that would impair the scheduled task action. If you deleted this file by accident, use the auto_LitFetchR_setup() to create a new task and a new code file. The script gives the name of the task to the run log and ends with a non-zero exit status if the run failed or finished with a problem.

Each time the task runs, the following files are created or updated:

• history_id.xlsx

  This file contains all the databases unique identifiers (e.g. PMID or SCOPUS_ID). These are saved and used at each reference retrieval to only extract the references that were not retrieved before. Identifiers are added only after the output file was written, and only for the references actually retrieved. A platform that failed, or a run that could not write its output, therefore never marks references as seen: they are retrieved at the next run.

• run_log.csv

  This file records every run, manual or scheduled, with one row per saved search and platform: the date and time, the versions of LitFetchR and R, the run mode (manual or scheduled:<task name>), the search string as read from “search_list.txt”, the exact query sent to the platform, the number of results and of new references, the status (success, no_results, partial or error) and the error message if any. For PubMed it also stores the query translation returned by the platform and for Web of Science the number of references per source database. Open it to check that your scheduled task is running and that no platform failed.

• citationsCSV_YYYY_MM_HHMMSS.CSV (or citationsRIS_YYYY_MM_HHMMSS.ris)

  This file contains the list of references retrieved from the search conducted at the date referenced in the name of the file. It is a CSV file, ready to be imported into the screening tool of your choice or in your reference manager. If the deduplication option was selected (dedup = TRUE), this file contains the unique dedupicated references. This file automatically opens after the reference retrieval ends if the corresponding option is selected (open_file = TRUE). When no reference is new, no file is written. The default format is CSV. Use output_format = "ris" in manual_fetch() or auto_LitFetchR_setup() for a RIS file, which tools such as Covidence require (the format of a task is fixed when it is created: remove the task and create it again to change it).

• history_dedup_YYYY_MM_HHMMSS.xlsx (if dedup = TRUE)

  This file contains the history of each step of the deduplication, using the package ASySD.

Congratulations! Your automated reference retrieval is ready to work!

Good to know about scheduled retrieval

• The task only runs while the computer is on and you are logged in. A run missed for this reason loses nothing: new references are found by their identifier and not by date, so the next run retrieves them.
• If one platform fails (for example an expired API key), the other platforms continue. The failed platform is marked error in “run_log.csv” and its references are retrieved at the next run.
• If you stop receiving files, open “run_log.csv” first. Rows with error or partial give the reason. A task that was disabled later is not detected by LitFetchR, check it in your task scheduler.
• Avoid running manual_fetch() in the same directory at the time a scheduled run is due.
• On Windows, the Task Scheduler limits the length of the command it stores. A project folder with a very long path (for example deep inside a synchronised cloud folder) is refused with a message saying so. Use a folder with a shorter path, for example C:/reviews/my_review.

See also

• Remove or change a scheduled task

• Manual reference retrieval