Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

filelock error when working with drake plan #1280

Closed
mattwarkentin opened this issue Jun 18, 2020 · 21 comments
Closed

filelock error when working with drake plan #1280

mattwarkentin opened this issue Jun 18, 2020 · 21 comments

Comments

@mattwarkentin
Copy link

Hi,

Seemingly out of nowhere I keep getting the following error when I try to run any function that acts on my drake plan (e.g. visualization):

plan <- drake_plan()
vis_drake_graph(plan)
Error in filelock::lock(private$lock_file) : 
  Cannot lock file: '/Matt/.drake/drake/history/lock': Operation not supported
Error in filelock::unlock(lock) : 
  VECTOR_ELT() can only be applied to a 'list', not a 'closure'

I had a working plan, then this error started showing up. I worked backwards to slowly try to figure out what the issue was by removing functions and targets in the plan. Even clearing the entire global environment, resetting R, and creating an empty plan doesn't work...

I honestly have no idea why this started happening. I have even tried deleting the .drake directory, resetting R, and trying again but it still happens. Any idea what this error means? How to fix it?

@mattwarkentin mattwarkentin changed the title filelock error when attempting to visualize plan filelock error when working with drake plan Jun 18, 2020
@wlandau
Copy link
Member

wlandau commented Jun 19, 2020

This is an instance of #1232. If you remove .drake/drake/history and set history = FALSE for the next make(), the error should go away. Unfortunately, I am having trouble reproducing this general issue myself, and it is coming from a package outside drake (filelock), so I think that's all we can do for now.

@wlandau wlandau closed this as completed Jun 19, 2020
@mattwarkentin
Copy link
Author

Thanks for the response, but I had actually never ran make() yet for this project. I am still defining functions and targets, visualizing the structure as I go.

I tried your suggestions of deleting .drake/drake/history, but I still get the same error when I simply load my functions (by sourceing R scripts), define my plan by running plan <- drake_plan(...), and try to run vis_drake_graph(plan) or outdated(plan). Same error as before.

This seems to be independent of make() as I have never used this yet. Any other suggestions?

@mattwarkentin
Copy link
Author

mattwarkentin commented Jun 19, 2020

I now ran make(plan, history = FALSE) and it made my targets seemingly without issue. Then I tried to readd() in a target value or visualize the network again and I keep getting the filelock error for both.

@wlandau
Copy link
Member

wlandau commented Jun 19, 2020

vis_drake_graph() might touch history because it goes through drake_config(). I am surprised that readd() is causing that, but it could be an easier way to narrow down the issue. Would you post a reproducible example and include your sessionInfo()?

@mattwarkentin
Copy link
Author

I can't really reproduce this issue. But I have seemingly narrowed it down to possibly a permissions issue. The original directory is located within a mounted volume on my Mac (network drive at my institute), and not the internal storage on my Mac.

For some reason when I copied the whole directory over to my local machine it worked...so could it be a permissions issue? Within the network drive, the project is in a folder that I have full read/write permissions so I can't imagine why I would be limited in any way.

I really am at a loss for why this doesn't work when the directory is opened from its normal location.

@mattwarkentin
Copy link
Author

Session Info
R version 4.0.0 (2020-04-24)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Catalina 10.15.5

Matrix products: default
BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRlapack.dylib

locale:
[1] en_CA.UTF-8/en_CA.UTF-8/en_CA.UTF-8/C/en_CA.UTF-8/en_CA.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods  
[7] base     

other attached packages:
 [1] ggtext_0.1.0      gghighlight_0.3.0 ggforce_0.3.1    
 [4] assertthat_0.2.1  progress_1.2.2    here_0.1         
 [7] fs_1.4.1          glue_1.4.1        rhdf5_2.32.0     
[10] keras_2.3.0.0     reticulate_1.16   lubridate_1.7.9  
[13] vroom_1.2.1       readxl_1.3.1      openxlsx_4.1.5   
[16] yardstick_0.0.6   workflows_0.1.1   tune_0.1.0       
[19] rsample_0.0.7     recipes_0.1.12    parsnip_0.1.1    
[22] infer_0.5.2       dials_0.0.7       scales_1.1.1     
[25] broom_0.5.6       tidymodels_0.1.0  forcats_0.5.0    
[28] stringr_1.4.0     dplyr_1.0.0       purrr_0.3.4      
[31] readr_1.3.1       tidyr_1.1.0       tibble_3.0.1     
[34] ggplot2_3.3.2     tidyverse_1.3.0   drake_7.12.2     
[37] dotenv_1.0.2      conflicted_1.0.4  testthat_2.3.2   
[40] devtools_2.3.0    usethis_1.6.1    

loaded via a namespace (and not attached):
  [1] tidyselect_1.1.0     lme4_1.1-23         
  [3] htmlwidgets_1.5.1    grid_4.0.0          
  [5] pROC_1.16.2          munsell_0.5.0       
  [7] base64url_1.4        codetools_0.2-16    
  [9] statmod_1.4.34       DT_0.13             
 [11] future_1.17.0        miniUI_0.1.1.1      
 [13] withr_2.2.0          colorspace_1.4-1    
 [15] filelock_1.0.2       knitr_1.28          
 [17] prompt_1.0.0         rstudioapi_0.11     
 [19] stats4_4.0.0         tensorflow_2.2.0    
 [21] bayesplot_1.7.2      listenv_0.8.0       
 [23] rstan_2.19.3         polyclip_1.10-0     
 [25] farver_2.0.3         bit64_0.9-7         
 [27] DiceDesign_1.8-1     txtq_0.2.0          
 [29] rprojroot_1.3-2      vctrs_0.3.1         
 [31] generics_0.0.2       ipred_0.9-9         
 [33] xfun_0.15            R6_2.4.1            
 [35] markdown_1.1         rstanarm_2.19.3     
 [37] lhs_1.0.2            promises_1.1.1      
 [39] nnet_7.3-14          gtable_0.3.0        
 [41] globals_0.12.5       processx_3.4.2      
 [43] timeDate_3043.102    rlang_0.4.6         
 [45] clisymbols_1.2.0     zeallot_0.1.0       
 [47] splines_4.0.0        inline_0.3.15       
 [49] reshape2_1.4.4       modelr_0.1.8        
 [51] tidytext_0.2.4       threejs_0.3.3       
 [53] crosstalk_1.1.0.1    backports_1.1.8     
 [55] httpuv_1.5.4         rsconnect_0.8.16    
 [57] tokenizers_0.2.1     gridtext_0.1.1      
 [59] tools_4.0.0          lava_1.6.7          
 [61] ellipsis_0.3.1       sessioninfo_1.1.1   
 [63] ggridges_0.5.2       Rcpp_1.0.4.6        
 [65] plyr_1.8.6           base64enc_0.1-3     
 [67] ps_1.3.3             prettyunits_1.1.1   
 [69] rpart_4.1-15         zoo_1.8-8           
 [71] haven_2.3.1          furrr_0.1.0         
 [73] magrittr_1.5         colourpicker_1.0    
 [75] reprex_0.3.0         GPfit_1.0-8         
 [77] SnowballC_0.7.0      whisker_0.4         
 [79] packrat_0.5.0        storr_1.2.1         
 [81] matrixStats_0.56.0   pkgload_1.1.0       
 [83] tidyposterior_0.0.3  hms_0.5.3           
 [85] shinyjs_1.1          mime_0.9            
 [87] xtable_1.8-4         tidypredict_0.4.5   
 [89] shinystan_2.5.0      tfruns_1.4          
 [91] gridExtra_2.3        rstantools_2.1.0    
 [93] compiler_4.0.0       crayon_1.3.4        
 [95] minqa_1.2.4          StanHeaders_2.21.0-5
 [97] htmltools_0.5.0      later_1.1.0.1       
 [99] RcppParallel_5.0.1   DBI_1.1.0           
[101] tweenr_1.0.1         dbplyr_1.4.4        
[103] MASS_7.3-51.6        boot_1.3-25         
[105] Matrix_1.2-18        cli_2.0.2           
[107] parallel_4.0.0       gower_0.2.1         
[109] igraph_1.2.5         pkgconfig_2.0.3     
[111] xml2_1.3.2           foreach_1.5.0       
[113] dygraphs_1.1.1.6     prodlim_2019.11.13  
[115] rvest_0.3.5          janeaustenr_0.1.5   
[117] callr_3.4.3          digest_0.6.25       
[119] cellranger_1.1.0     shiny_1.4.0.2       
[121] gtools_3.8.2         nloptr_1.2.2.1      
[123] lifecycle_0.2.0      nlme_3.1-148        
[125] jsonlite_1.6.1       Rhdf5lib_1.10.0     
[127] desc_1.2.0           fansi_0.4.1         
[129] pillar_1.4.4         lattice_0.20-41     
[131] loo_2.2.0            fastmap_1.0.1       
[133] httr_1.4.1           pkgbuild_1.0.8      
[135] survival_3.2-3       xts_0.12-0          
[137] remotes_2.1.1        zip_2.0.4           
[139] shinythemes_1.1.2    iterators_1.0.12    
[141] bit_1.1-15.2         class_7.3-17        
[143] stringi_1.4.6        blob_1.2.1          
[145] memoise_1.1.0

@wlandau
Copy link
Member

wlandau commented Jun 22, 2020

Thanks. And next time you see the error, would you post a traceback too?

@wlandau wlandau reopened this Jun 22, 2020
@mattwarkentin
Copy link
Author

## Load packages ----
source("_packages.R")

## Source R files ----
purrr::walk(fs::dir_ls("./R", glob = "*.R"), source)

vis_drake_graph(plan)
19: filelock::unlock(lock)
18: private$txtq_exclusive({
        file_create(private$db_file)
        if (!file.exists(private$head_file)) {
            private$txtq_set_head(0)
        }
        if (!file.exists(private$total_file)) {
            private$txtq_set_total(0)
        }
    })
17: private$txtq_establish(path)
16: .subset2(public_bind_env, "initialize")(...)
15: R6_txtq$new(path = path)
14: txtq::txtq(history_path)
13: history_queue(history_path)
12: recover_default_history(path)
11: methods::initRefFields(.Object, classDef, selfEnv, list(...))
10: initialize(value, ...)
9: initialize(value, ...)
8: methods::new(def, ...)
7: refclass_decorated_storr$new(storr = storr, driver = storr$driver, 
       default_namespace = storr$default_namespace, envir = storr$envir, 
       hash_algorithm = hash_algorithm, digest = digest, history = recover_default_history(path), 
       ht_encode_path = ht_new(), ht_decode_path = ht_new(), ht_encode_namespaced = ht_new(), 
       ht_decode_namespaced = ht_new(), ht_hash = ht_new(), ht_keys = ht_keys(digest), 
       path = path, path_return = file.path(path, "drake", "return"), 
       path_tmp = file.path(path, "drake", "tmp"))
6: decorate_storr(storr::storr_rds(path = path))
5: drake_fetch_rds(path = path)
4: this_cache_(path = path)
3: drake::drake_cache()
2: drake_config2(...)
1: vis_drake_graph(plan)

@wlandau
Copy link
Member

wlandau commented Jun 22, 2020

Okay, that really helps. Does txtq::txtq(".drake/drake/history") give the same error? If so, would you be willing to upload that history folder here as a zip archive? In the meantime, I will try a workaround in the txtq package, which I also maintain.

@mattwarkentin
Copy link
Author

txtq::txtq(".drake/drake/history")
Error in filelock::lock(private$lock_file) : 
  Cannot lock file: '/Volumes/hung_lab/shared/Matt/deep-learning/.drake/drake/history/lock': Operation not supported
Error in filelock::unlock(lock) : 
  VECTOR_ELT() can only be applied to a 'list', not a 'closure'
6: filelock::unlock(lock)
5: private$txtq_exclusive({
       file_create(private$db_file)
       if (!file.exists(private$head_file)) {
           private$txtq_set_head(0)
       }
       if (!file.exists(private$total_file)) {
           private$txtq_set_total(0)
       }
   })
4: private$txtq_establish(path)
3: .subset2(public_bind_env, "initialize")(...)
2: R6_txtq$new(path = path)
1: txtq::txtq(".drake/drake/history")

@mattwarkentin
Copy link
Author

I will upload the history folder, one sec.

@mattwarkentin
Copy link
Author

I think this is what you wanted. This is a zip of .drake/drake/history

history.zip

@wlandau
Copy link
Member

wlandau commented Jun 22, 2020

Thanks for that. Odd, but I cannot reproduce the error with txtq::txtq("history") with either development txtq or CRAN txtq. What happens if you run this?

library(filelock)
lock <- lock("history/lock")
unlock(lock)

@wlandau
Copy link
Member

wlandau commented Jun 22, 2020

Also worth checking file.access("history/lock", mode = x) for x = 0, 1, 2, and 4. If for some reason you do not have permission to write the file, that might cause problems.

@mattwarkentin
Copy link
Author

mattwarkentin commented Jun 22, 2020

library(filelock)
lock <- lock(".drake/drake/history/lock")
unlock(lock)
Error in lock(".drake/drake/history/lock") : 
  Cannot lock file: '/Volumes/hung_lab/shared/Matt/deep-learning/.drake/drake/history/lock': Operation not supported
Error in unlock(lock) : 
  VECTOR_ELT() can only be applied to a 'list', not a 'closure'

Regarding permission:

fs::file_access(".drake/drake/history/lock", c("read", "write", "execute"))
.drake/drake/history/lock 
                     TRUE 

@wlandau
Copy link
Member

wlandau commented Jun 22, 2020

I think we have narrowed down the issue as much as we can here. I think the next thing to do is ask Gabor about it on the filelock issue tracker. Would you post a new issue to https://github.com/r-lib/filelock and cc me?

By the way, thank you so much for helping us get to the bottom of this issue. I have heard other reports of this happening (which sometimes could be avoided by suppressing history) but it is super tough to reproduce on my end.

@mattwarkentin
Copy link
Author

No problem, I'll post the issue over on filelock GitHub right now.

Not a problem; happy to help any way I can. Thanks a lot for your help. I am fairly new to {drake} which makes me feel even more helpless when this error showed up. Thanks for being so responsive and working toward a solution.

@wlandau
Copy link
Member

wlandau commented Jun 23, 2020

You know what? After reading the discussions of file systems at r-lib/filelock#26, I think I need to revisit wlandau/txtq#18.

@wlandau
Copy link
Member

wlandau commented Jun 23, 2020

NB the fix requires txtq version 0.2.1, which is only on GitHub now (remotes::install_github("wlandau/txtq")).

@wlandau
Copy link
Member

wlandau commented Jun 23, 2020

@pydupont confirmed the patch works, so I am closing. Please let me know if it persists for you.

@mattwarkentin
Copy link
Author

@wlandau I also confirm everything seems to be working perfectly after installing the dev versions of txtq and drake. Thanks so much for all of your hard work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants