iBazel should tell tell Bazel what files changed

[filipesilva]

Currently both Bazel and iBazel watch the file system for changes. This can lead to a lot of duplicated file watchers.

If iBazel could take over watching sourcefiles and tell Bazel exactly which ones changed, Bazel itself would not have to watch those files.

cc @alexeagle

[achew22]

@filipesilva, I just looked through the flags in Bazel for something that would allow this, but I can't find anything. Can you point me to some docs or code that describes how to pass in the list of changed files to Bazel?

[filipesilva]

@achew22 I'm not sure if such a thing exists yet actually. This item came out of a discussion on how to have iBazel and Bazel coexisting without overlap in file watchers.

[achew22]

Hrmm, maybe we should create an issue over on the Bazel repo then. I don't think there is any action to be performed in iBazel before something changes over there.

Maybe we could use this issue to understand why you have so many files that are being watched in the first place. Are you hitting your OS limit? We have code in iBazel that should crank up the ulimit to the OS max, are you exceeding even that? Could you help me understand the contours of your repo? Do you have a huge repo or a large number of deps? Are your deps venored or imported through repository rules?

[filipesilva]

Heya @achew22, sorry it took me so long to get back to you. I agree that this feature request needs support from Bazel directly.

The repository in question is https://github.com/angular/angular. But to be honest this is more of a conceptual problem where whatever the total file system watcher limit is, it's effective number will be halved if both Bazel and iBazel are watching the same things.

In the NodeJS ecosystem it's very common to have many small external module dependencies, so it's common enough to hit the (default) file system limit.

I'm not sure there is an immediate action to take. But I wanted to capture this context for if/when this problem starts popping up.

[achew22]

@filipesilva, this issue may have been rendered moot. IBazel's watcher system no longer watches individual files and instead watches directories. When you combine that with the new Bazel managed dependency system for Typescript (node_modules is loaded by a repository rule) I would expect the number of watchers needed to be O(hundreds) for most projects and O(thousands) for a very large project like angular/angular.

Have you been having problems lately?

[filipesilva]

No, I have not been having problems lately. I agree that this issue is mostly moot with the changes you mentioned, and thus it is better to close it. Cheers!