Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

drake_debug does not work with dynamic targets #1214

Closed
3 tasks done
kendonB opened this issue Mar 11, 2020 · 7 comments
Closed
3 tasks done

drake_debug does not work with dynamic targets #1214

kendonB opened this issue Mar 11, 2020 · 7 comments
Assignees

Comments

@kendonB
Copy link
Contributor

kendonB commented Mar 11, 2020

Prework

  • Read and abide by drake's code of conduct.
  • Search for duplicates among the existing issues, both open and closed.
  • Advanced users: verify that the bug still persists in the current development version (i.e. remotes::install_github("ropensci/drake")) and mention the SHA-1 hash of the Git commit you install. ropensci/drake@36b978e

Description

Trying to use drake_debug on a dynamic target seems to not work properly. I have tried both the target itself as well as a subtarget.

Reproducible example

library(broom)
library(drake)
library(gapminder)
library(tidyverse)

# Split the Gapminder data by continent.
gapminder_continents <- function() {
  gapminder %>%
    mutate(gdpPercap = scale(gdpPercap)) %>%
    split(f = .$continent)
}

# Fit a model to a continent.
fit_model <- function(continent_data) {
  data <- continent_data[[1]]
  data %>%
    lm(formula = gdpPercap ~ year) %>%
    tidy() %>%
    mutate(continent = data$continent[1]) %>%
    select(continent, term, statistic, p.value)
}

plan <- drake_plan(
  continents = gapminder_continents(),
  model = target(fit_model(continents), dynamic = map(continents))
)
make(plan, target = "continents")
#> ▶ target continents
drake_debug(model, plan = plan)
#> ℹ Building target  model  in debug mode.
#> ▶ dynamic model

### MANUALLY ADDED TO REPREX ####

drake_debug(model_53c1b086, plan = plan) 
#> ℹ Building target  model_53c1b086  in debug mode. 
#> ▶ target model_53c1b086 
#> debugging in: f() 
#> debug at <text>#1: { 
#>     NULL 
#> }
#> Browse[2]>

Created on 2020-03-12 by the reprex package (v0.3.0)

Session info
devtools::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value                       
#>  version  R version 3.6.3 (2020-02-29)
#>  os       Ubuntu 18.04.4 LTS          
#>  system   x86_64, linux-gnu           
#>  ui       X11                         
#>  language en_US                       
#>  collate  en_NZ.UTF-8                 
#>  ctype    en_NZ.UTF-8                 
#>  tz       Pacific/Auckland            
#>  date     2020-03-12                  
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package     * version     date       lib source                         
#>  assertthat    0.2.1       2019-03-21 [1] CRAN (R 3.6.1)                 
#>  backports     1.1.5       2019-10-02 [1] CRAN (R 3.6.1)                 
#>  base64url     1.4         2018-05-14 [1] CRAN (R 3.6.1)                 
#>  broom       * 0.5.2       2019-04-07 [1] CRAN (R 3.6.1)                 
#>  callr         3.4.0       2019-12-09 [1] CRAN (R 3.6.1)                 
#>  cellranger    1.1.0       2016-07-27 [1] CRAN (R 3.6.1)                 
#>  cli           2.0.0       2019-12-09 [1] CRAN (R 3.6.1)                 
#>  colorspace    1.4-1       2019-03-18 [1] CRAN (R 3.6.1)                 
#>  crayon        1.3.4       2017-09-16 [1] CRAN (R 3.6.1)                 
#>  DBI           1.0.0       2018-05-02 [1] CRAN (R 3.6.1)                 
#>  dbplyr        1.4.2       2019-06-17 [1] CRAN (R 3.6.1)                 
#>  desc          1.2.0       2018-05-01 [1] CRAN (R 3.6.1)                 
#>  devtools      2.2.1       2019-09-24 [1] CRAN (R 3.6.1)                 
#>  digest        0.6.25      2020-02-23 [1] CRAN (R 3.6.3)                 
#>  dplyr       * 0.8.3       2019-07-04 [1] CRAN (R 3.6.1)                 
#>  drake       * 7.11.0.9000 2020-03-11 [1] github (ropensci/drake@36b978e)
#>  ellipsis      0.3.0       2019-09-20 [1] CRAN (R 3.6.1)                 
#>  evaluate      0.14        2019-05-28 [1] CRAN (R 3.6.1)                 
#>  fansi         0.4.0       2018-10-05 [1] CRAN (R 3.6.1)                 
#>  filelock      1.0.2       2018-10-05 [1] CRAN (R 3.6.1)                 
#>  forcats     * 0.4.0       2019-02-17 [1] CRAN (R 3.6.1)                 
#>  fs            1.3.1       2019-05-06 [1] CRAN (R 3.6.1)                 
#>  gapminder   * 0.3.0       2017-10-31 [1] CRAN (R 3.6.3)                 
#>  generics      0.0.2       2018-11-29 [1] CRAN (R 3.6.1)                 
#>  ggplot2     * 3.2.1       2019-08-10 [1] CRAN (R 3.6.1)                 
#>  glue          1.3.1       2019-03-12 [1] CRAN (R 3.6.1)                 
#>  gtable        0.3.0       2019-03-25 [1] CRAN (R 3.6.1)                 
#>  haven         2.2.0       2019-11-08 [1] CRAN (R 3.6.1)                 
#>  highr         0.8         2019-03-20 [1] CRAN (R 3.6.1)                 
#>  hms           0.5.2       2019-10-30 [1] CRAN (R 3.6.1)                 
#>  htmltools     0.4.0       2019-10-04 [1] CRAN (R 3.6.1)                 
#>  httr          1.4.1       2019-08-05 [1] CRAN (R 3.6.1)                 
#>  igraph        1.2.4.2     2019-11-27 [1] CRAN (R 3.6.1)                 
#>  jsonlite      1.6         2018-12-07 [1] CRAN (R 3.6.1)                 
#>  knitr         1.26        2019-11-12 [1] CRAN (R 3.6.1)                 
#>  lattice       0.20-38     2018-11-04 [1] CRAN (R 3.5.1)                 
#>  lazyeval      0.2.2       2019-03-15 [1] CRAN (R 3.6.1)                 
#>  lifecycle     0.1.0       2019-08-01 [1] CRAN (R 3.6.1)                 
#>  lubridate     1.7.4       2018-04-11 [1] CRAN (R 3.6.1)                 
#>  magrittr      1.5         2014-11-22 [1] CRAN (R 3.6.1)                 
#>  memoise       1.1.0       2017-04-21 [1] CRAN (R 3.6.1)                 
#>  modelr        0.1.5       2019-08-08 [1] CRAN (R 3.6.1)                 
#>  munsell       0.5.0       2018-06-12 [1] CRAN (R 3.6.1)                 
#>  nlme          3.1-142     2019-11-07 [1] CRAN (R 3.6.1)                 
#>  pillar        1.4.2       2019-06-29 [1] CRAN (R 3.6.1)                 
#>  pkgbuild      1.0.6       2019-10-09 [1] CRAN (R 3.6.1)                 
#>  pkgconfig     2.0.3       2019-09-22 [1] CRAN (R 3.6.1)                 
#>  pkgload       1.0.2       2018-10-29 [1] CRAN (R 3.6.1)                 
#>  prettyunits   1.0.2       2015-07-13 [1] CRAN (R 3.6.1)                 
#>  processx      3.4.1       2019-07-18 [1] CRAN (R 3.6.1)                 
#>  progress      1.2.2       2019-05-16 [1] CRAN (R 3.6.1)                 
#>  ps            1.3.0       2018-12-21 [1] CRAN (R 3.6.1)                 
#>  purrr       * 0.3.3       2019-10-18 [1] CRAN (R 3.6.1)                 
#>  R6            2.4.1       2019-11-12 [1] CRAN (R 3.6.1)                 
#>  Rcpp          1.0.3       2019-11-08 [1] CRAN (R 3.6.1)                 
#>  readr       * 1.3.1       2018-12-21 [1] CRAN (R 3.6.1)                 
#>  readxl        1.3.1       2019-03-13 [1] CRAN (R 3.6.1)                 
#>  remotes       2.1.0       2019-06-24 [1] CRAN (R 3.6.1)                 
#>  renv          0.9.1-1     2019-12-10 [1] github (rstudio/renv@5f0234e)  
#>  reprex        0.3.0       2019-05-16 [1] CRAN (R 3.6.1)                 
#>  rlang         0.4.2.9000  2019-12-04 [1] github (r-lib/rlang@1be25e7)   
#>  rmarkdown     1.18        2019-11-27 [1] CRAN (R 3.6.1)                 
#>  rprojroot     1.3-2       2018-01-03 [1] CRAN (R 3.6.1)                 
#>  rvest         0.3.5       2019-11-08 [1] CRAN (R 3.6.1)                 
#>  scales        1.1.0       2019-11-18 [1] CRAN (R 3.6.1)                 
#>  sessioninfo   1.1.1       2018-11-05 [1] CRAN (R 3.6.1)                 
#>  storr         1.2.1       2018-10-18 [1] CRAN (R 3.6.1)                 
#>  stringi       1.4.3       2019-03-12 [1] CRAN (R 3.6.1)                 
#>  stringr     * 1.4.0       2019-02-10 [1] CRAN (R 3.6.1)                 
#>  testthat      2.3.1       2019-12-01 [1] CRAN (R 3.6.1)                 
#>  tibble      * 2.1.3       2019-06-06 [1] CRAN (R 3.6.1)                 
#>  tidyr       * 1.0.0       2019-09-11 [1] CRAN (R 3.6.1)                 
#>  tidyselect    0.2.5       2018-10-11 [1] CRAN (R 3.6.1)                 
#>  tidyverse   * 1.3.0       2019-11-21 [1] CRAN (R 3.6.1)                 
#>  txtq          0.2.0       2019-10-15 [1] CRAN (R 3.6.1)                 
#>  usethis       1.5.1       2019-07-04 [1] CRAN (R 3.6.1)                 
#>  vctrs         0.2.0.9007  2019-12-10 [1] github (r-lib/vctrs@a3c4b54)   
#>  withr         2.1.2       2018-03-15 [1] CRAN (R 3.6.1)                 
#>  xfun          0.11        2019-11-12 [1] CRAN (R 3.6.1)                 
#>  xml2          1.2.2       2019-08-09 [1] CRAN (R 3.6.1)                 
#>  yaml          2.2.0       2018-07-25 [1] CRAN (R 3.6.1)                 
#> 
#> [1] /media/bellk/DATADRIVE/Dropbox/projects_ac/climate_nz/renv/library/R-3.6/x86_64-pc-linux-gnu
#> [2] /tmp/RtmpaYK23g/renv-system-library
#> [3] /tmp/Rtmpmh5Exq/renv-system-library

Expected result

What should have happened? Please be as specific as possible.

Session info

End the reproducible example with a call to sessionInfo() in the same session (e.g. reprex(si = TRUE)) and include the output.

@wlandau
Copy link
Member

wlandau commented Mar 12, 2020

For drake_debug() and drake_build(), I have to draw the line at static targets. The technical debt is not worth the user-side convenience. The help files already say these functions are not valid for dynamic targets, but we probably need informative errors.

@kendonB
Copy link
Contributor Author

kendonB commented Mar 12, 2020

Sorry I did not see that in the docs! I'm just wondering if there's a good method for debugging a specific subtarget. All you know about the subtarget that is giving you trouble is the name as far as I can tell. How do you back out which element of the grouping variable(s) produces that subtarget?

@wlandau
Copy link
Member

wlandau commented Mar 12, 2020

To be honest, I am not sure, and I have been having the same trouble in a large drake workflow at my day job over the past couple days. One natural idea is to save the elements you map()ped over for the sub-target so you can retrieve them with diagnose(), but I hesitate to do that because they could get quite large. Maybe just the sub-target index?

@wlandau
Copy link
Member

wlandau commented Mar 12, 2020

If we always store the sub-target index, that might even replace the trace.

@wlandau
Copy link
Member

wlandau commented Mar 12, 2020

Ignore the last 2 comments, I got mixed up. Here is a debugging technique that should help you locate a particular sub-target on a grid of map()ped or cross()ed grouping variables. It might be a good idea to support a wrapper subtargets_predicted() around the internal subtarget_names() for debugging purposes, not sure yet though.

library(drake)
plan <- drake_plan(
  nums = 1:2,
  result = target(
    stopifnot(nums <= 1L),
    dynamic = map(nums)
  )
)

make(plan)
#> ▶ target nums
#> ▶ dynamic result
#> > subtarget result_0b3474bd
#> > subtarget result_b2a5c9b8
#> x fail result_b2a5c9b8
#> Error: target result_b2a5c9b8 failed.
#> diagnose(result_b2a5c9b8)error$message:
#>   nums <= 1L is not TRUE
#> diagnose(result_b2a5c9b8)error$calls:
#>   1. └─base::eval(expr = tidy_expr_8a6af5, envir = config$envir_subtargets)
#>   2.   ├─base::local(stopifnot(nums <= 1L))
#>   3.   │ └─base::eval.parent(substitute(eval(quote(expr), envir)))
#>   4.   │   └─base::eval(expr, p)
#>   5.   │     └─base::eval(expr, p)
#>   6.   └─base::eval(quote(stopifnot(nums <= 1L)), new.env())
#>   7.     └─base::eval(quote(stopifnot(nums <= 1L)), new.env())
#>   8.       └─base::stopifnot(nums <= 1L)

config <- drake_config(plan)

sub <- drake:::subtarget_names("result", config)
fld <- failed()

sub
#> [1] "result_0b3474bd" "result_b2a5c9b8"

fld
#> [1] "result"          "result_b2a5c9b8"

# Index in the grid
which(sub %in% fld)
#> [1] 2

Created on 2020-03-12 by the reprex package (v0.3.0)

@kendonB
Copy link
Contributor Author

kendonB commented Mar 13, 2020

Great this works - seems worth making a wrapper for this? Like which_subtarget_failed(dynamic_target, plan)?

It would return a named vector of indices for cross and a single index for map?

@wlandau
Copy link
Member

wlandau commented Mar 14, 2020

On second thought, an API function for this niche use case seems kind of heavy handed. In c211885, I added the sub-target name and index to the error metadata of the dynamic sub-target and its parent.

library(drake)

plan <- drake_plan(
  nums = seq_len(2L),
  result = target(
    stopifnot(nums < 2L),
    dynamic = map(nums)
  )
)

make(plan)
#> ▶ target nums
#> ▶ dynamic result
#> > subtarget result_0b3474bd
#> > subtarget result_b2a5c9b8
#> x fail result_b2a5c9b8
#> Error: target result_b2a5c9b8 failed.
#> diagnose(result_b2a5c9b8)error$message:
#>   nums < 2L is not TRUE
#> diagnose(result_b2a5c9b8)error$calls:
#>   1. └─base::eval(expr = tidy_expr_8a6af5, envir = config$envir_subtargets)
#>   2.   ├─base::local(stopifnot(nums < 2L))
#>   3.   │ └─base::eval.parent(substitute(eval(quote(expr), envir)))
#>   4.   │   └─base::eval(expr, p)
#>   5.   │     └─base::eval(expr, p)
#>   6.   └─base::eval(quote(stopifnot(nums < 2L)), new.env())
#>   7.     └─base::eval(quote(stopifnot(nums < 2L)), new.env())
#>   8.       └─base::stopifnot(nums < 2L)

f <- failed()
f
#> [1] "result"          "result_b2a5c9b8"

diagnose(f[1], character_only = TRUE)
#> $error
#> <simpleError in eval(quote(stopifnot(nums < 2L)), new.env()): nums < 2L is not TRUE>
#> 
#> $seed
#> [1] 1920294608
#> 
#> $subtarget
#> [1] "result_b2a5c9b8"
#> 
#> $subtarget_index
#> [1] 2

diagnose(f[2], character_only = TRUE)
#> $error
#> <simpleError in eval(quote(stopifnot(nums < 2L)), new.env()): nums < 2L is not TRUE>
#> 
#> $seed
#> [1] 1920294608
#> 
#> $subtarget
#> [1] "result_b2a5c9b8"
#> 
#> $subtarget_index
#> [1] 2

Created on 2020-03-13 by the reprex package (v0.3.0)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants