Import metadata

[ghost]

I haven't found a REDCapR function that'll allow me to update/import metadata (i.e. data dictionary) to a project. I know that I can read the data dictionary with:

REDCapR::redcap_metadata_read(redcap_uri, token)$data

But to my knowledge, a function that does the opposite does not exist. How would one accomplish this?

[ghost]

So far, I've tried editing redcap_write_oneshot() to:

redcap_write_oneshot_metadata <- function(
  ds,
  redcap_uri,
  token,
  verbose         = TRUE,
  config_options  = NULL
) {
...
post_body <- list(
    token     = token,
    content   = "metadata", # instead of content = 'record'
    format    = "csv",
    type      = "flat",
    data                = csv,
    overwriteBehavior   = "overwrite",
    returnContent       = "ids",
    returnFormat        = "csv"
  )
...
}

But this doesn't seem to do the job and I'm at a loss.

[wibeasley]

@felixetorres, I'm reluctant to add this as a real function in REDCapR because (a) it requires a supertoken, (b) I want all real functions to have a unit test, and (c) I don't want to deploy a supertoken to Travis-CI that would test the function.

But I'll gladly help you write a snippet that can do this. I just don't want it to be a supported feature of REDCapR. Let's something like in this participantList snippet which uses the private-ish REDCapR:::kernel_api().

Note that kernel_api() requires three semicolons (because it's not publicly exported --at least not currently).

config_options <- NULL
uri            <- "https://qqqqq.edu/redcap/api/"
token          <- "qqqqqqqqq"
post_body      <- list(
  token      = token,
  content    = 'metadata',
  format     = 'json',
  # data       = '',
  data       = '"Variable / Field Name","Form Name","Section Header","Field Type","Field Label","Choices, Calculations, OR Slider Labels","Field Note","Text Validation Type OR Show Slider Number","Text Validation Min","Text Validation Max",Identifier?,"Branching Logic (Show field only if...)","Required Field?","Custom Alignment","Question Number (surveys only)","Matrix Group Name","Matrix Ranking?","Field Annotation"\nrecord_id,demographics,,text,"Study ID",,,,,,,,,,,,,\nname_first,demographics,"Contact Information",text,"First Name",,,,,,y,,,,,,,\nname_last,demographics,,text,"Last Name",,,,,,y,,,,,,,\naddress,demographics,,notes,"Street, City, State, ZIP",,,,,,y,,,,,,,\ntelephone,demographics,,text,"Phone number",,"Include Area Code",phone,,,y,,,,,,,\nemail,demographics,,text,E-mail,,,email,,,y,,,,,,,\ndob,demographics,,text,"Date of birth",,,date_ymd,,,y,,,,,,,\nage,demographics,,text,"Age (years)",,,,,,,,,,,,,\nsex,demographics,,radio,Gender,"0, Female | 1, Male",,,,,,,,,,,,\nheight,health,,text,"Height (cm)",,,number,130,215,,,,,,,,\nweight,health,,text,"Weight (kilograms)",,,integer,35,200,,,,,,,,\nbmi,health,,calc,BMI,"round(([weight]*10000)/(([height])^(2)),1)",,,,,,,,,,,,\ncomments,health,"General Comments",notes,Comments,,,,,,,,,,,,,\nmugshot,health,,file,Mugshot,,,,,,,,,,,,,\nrace,race_and_ethnicity,,checkbox,"Race (Select all that apply)","1, American Indian/Alaska Native | 2, Asian | 3, Native Hawaiian or Other Pacific Islander | 4, Black or African American | 5, White | 6, Unknown / Not Reported",,,,,,,,,,,,\nethnicity,race_and_ethnicity,,radio,Ethnicity,"0, Unknown / Not Reported | 1, NOT Hispanic or Latino | 2, Hispanic or Latino",,,,,,,,,,,,\n'
  format     = 'csv'
)
kernel <- REDCapR:::kernel_api(uri, post_body, config_options)

I don't have super tokens enabled on my machine, so I can't test this. Does this work?

edit: added the csv to the content parameter.

[ghost]

Thanks so much for your response @wibeasley !

Here's what I get when I reproduce this:

0 records were written to REDCap in 0.1 seconds.

And the listed output is:

$success
[1] TRUE

$status_code
[1] 200

$outcome_message
[1] "0 records were written to REDCap in 0.1 seconds."

$records_affected_count
[1] 0

$affected_ids
character(0)

$elapsed_seconds
[1] 0.1183209

$raw_text
[1] ""

Even though I have records in the data dictionary file in the argument I'm supplying. Could it be because the data needs to be in JSON format, since you specified:

post_body      <- list(
  token      = token,
  content    = 'metadata',
  format     = 'json', ########### this?
  data       = data_input,
  format     = 'csv'
)

To clarify: the code did work in the sense that it wrote out data = ''to Redcap, but the data dictionary is now blank, even when I supply data = csv, for example.

[wibeasley]

@felixetorres, sorry, I should have followed through the example all the way. You're right, the dictionary belongs in the content parameter. I just edited my previous post with an explicit CSV. (I don't think the csv vs json matters much, because the dictionary is a nice rectangle.)

In your real code, you'll probably want to pull the dictionary from a file. Consider readr::read_file(), which doesn't care if the format is csv, xml, or json.

[ghost]

@wibeasley Thank you so much! Your original code actually worked flawlessly (as long as I supplied the data into content), but I just interpreted the results wrong. I'm not sure how a supertoken is different than the other tokens that are supplied to your functions, but I hope there's a secure/easy/efficient way to perform this with one of your functions in the future. Cheers!

[wibeasley]

@felixetorres, you're right, a supertoken isn't required to Import Metadata. A supertoken is required to create a REDCap project, and I incorrectly applied that to defining a project too.

So thanks for convincing me. I'll add this function and the accompanying tests. ...unless you'd like to take a shot at a PR? The structure would be very similar to

• R/redcap-metadata-read.R, and
• tests/testthat/test-metadata-read.R

The test project could by pid 213 in the credential file. That's already tasked with allowing the test suite to delete the project and write new rows.

(I have to say that I'm not entirely comfortable know that someone that I gave read/write permission can also modify the variables. But that's not something REDCapR can protect since the permissions are at the server-level, not the client-level. If anything, a dedicated REDCapR function would reduce the chance that someone would accidentally pass the wrong parameters to curl and mess with the variables.)

[ghost]

That's a good point- I didn't think about someone incorrectly modifying variables, especially if there's only one copy of the Redcap project. (My group usually makes 2 copies, but that's beside the point.)

Happy to submit a PR in case you decide to include this function, though. Thanks!

[SamGuay]

Great to see this being done! I was trying to come up with something similar on my own and I finally found this issue. Looking forward to seeing it merged!

[wibeasley]

Thanks, @felixetorres for the code. I pulled it into the dev and made a few minor tweaks first (PR #284). I'm about to pull it into the master (PR #289). Please tell me if the function still fits your needs.

@SamGuay, if you have suggestions or advice, I'd like to hear about them in a new issue (and please reference this one).