Feature request: support multiple source entry regex paths

[jeromedalbert]

To mirror Rails' ability to add additional assets in config/initializers/assets.rb, it would be nice if Webpacker could do the same for entry paths.

For example in one of my apps, I have the following asset configuration:

# config/initializers/assets.rb

Rails.application.config.assets.precompile += %w(
  **/application.css
  **/pages/*.css
  **/main.css
)

The file above used to precompile JS on top of CSS. But now that I have moved all my JavaScript "sprinkle code" over to Webpack (because I want all the app to use next generation JavaScript), I needed a similar feature for Webpacker.

Here's what I have done to make this work:

# config/webpacker.yml

default: &default
  source_path: app/javascript
  source_entries: # I have removed source_entry_path and replaced it with "source_entries"
    - '**/application'
    - '**/pages/*'
    - '**/main'

  ...

// config/webpack/shared.js

...

let packPaths = [];
settings.source_entries.forEach((sourceEntry) => {
  const path = `${settings.source_path}/${sourceEntry}{${settings.extensions.join(',')}}*`;
  packPaths = packPaths.concat(sync(path));
});
const sourceEntries = packPaths.reduce((map, path) => {
  const namespace = relative(settings.source_path, dirname(path));
  map[join(namespace, basename(path, extname(path)))] = [resolve(path)];
  return map;
}, {});

module.exports = {
  entry: sourceEntries,

  ...

That's it. I think this could be a common enough use case, especially in apps that bundle one CSS stylesheet and one JS file per page, like the one I am currently working on:

<!-- Example typical page declaring its dependencies at the top -->

<% content_for :stylesheets do %>
  <%= stylesheet_link_tag 'marketing_website/pages/home', media: 'all' %>
<% end %>

<% content_for :javascripts do %>
  <%= javascript_pack_tag 'marketing_website/pages/home' %>
<% end %>

<section>
  <article>
   ...

Do you guys think it would be useful to have this feature in Webpacker? I could do a PR with the modifications above.

[gauravtiwari]

@jeromedalbert You could mirror same by requiring these files into a webpacker pack like so:

// app/javascript/packs/marketing/home.js
import '../../assets/css/marketing.css'

// .... rest of the code

// app/javascript/packs/app/home.js
import '../../assets/css/app.css'

// .... rest of the code

<% content_for :stylesheets do %>
  <%= stylesheet_pack_tag 'marketing/home', media: 'all' %>
<% end %>

<% content_for :stylesheets do %>
  <%= stylesheet_pack_tag 'app/home', media: 'all' %>
<% end %>

<% content_for :javascripts do %>
  <%= javascript_pack_tag 'marketing/home' %>
<% end %>

<% content_for :javascripts do %>
  <%= javascript_pack_tag 'app/home' %>
<% end %>

What do you think?

[jeromedalbert]

Right, in my case I have only migrated the JS files to Webpacker, and kept the CSS/images/fonts in the asset pipeline, since I had no particular reason to migrate them so far.

But even if I migrated the CSS all the way, your suggestion implies that I have to type my CSS declarations twice. First with <%= stylesheet_pack_tag 'marketing/home', media: 'all' %>, and then with import '../../assets/css/marketing.css'. So this is twice more boilerplate than my current implementation where I just write <%= stylesheet_link_tag 'marketing_website/pages/home', media: 'all' %> which points to the right asset thanks to the assets.rb glob.

As for the JS files, if I am not mistaken, it looks like the convention is to use app/javascript/packs/* files as references to app/javascript/* folders. For example app/javascript/packs/calendar.js's only content is import 'calendar', which loads app/javascript/calendar/index.js. In my case, each page (or most pages) would have their own Webpack-compiled JS "main" or index file. Having as many packs as you have pages in your apps would be tedious to code because it increases the amount of boilerplate/indirections (same idea as the CSS paragraph above). So I don't use the packs folder, and directly write files+folders in app/javascript/*, where the ones named application.js/main.js/etc are correctly picked up by javascript_pack_tag thanks to the glob in my previous comment. (Note: I could also put all my files+folders in app/javascript/packs, but that would make for longer paths, and I would still need to use a glob for subfolders).

Now I realize that organizing JS files my way is not mentioned in the Webpacker README at all. I just thought that since the Rails asset pipeline already allowed such organizations (with minimum boilerplate) thanks to assets.rb globs, maybe Webpacker could have a similar option (with minimum boilerplate, i.e. thanks to node's glob in the Webpacker config).

Sorry if I was unclear, I can explain further if needed. Also, if I am only one of the rare weirdos with this kind of JS/CSS organization, we can just close this issue and keep it as a reference for a future hypothetical other weirdo. 😄

[mach-kernel]

+1 for this. I don't need any of the regex portions of this, but I would like to have multiple entry points. For example, our organization has a style guide where our designer creates components for our main application. We pull this in through a submodule and have other SPA-like components independent from the main style guide, if not just external assets. @jeromedalbert, do you have a branch for this? Would love to check it out and provide you with some feedback / maybe collaborate on making this into a PR.

[jeromedalbert]

@mach-kernel Yeah, here is my branch. It supports the existing source_entry_path key (accepts one value), as well as a new source_entry_paths key (accepts an array).

Although for your use case, since you don't need regex globs, you could make it work with current Webpacker by just adding one file per entry point in app/javascript/packs (i.e. app/javascript/packs/your_entry_point1 and app/javascript/packs/your_entry_point2). Unless I misunderstood something.

[mach-kernel]

@jeromedalbert thanks for the branch! The reason would be mainly to keep the two entry points in different directories. I want to keep my Git submodule at the root of my project to make it easier to access (again, not a hard breaking issue, but "in a perfect world it would be nice if"-kind of thing). 😺

[jeromedalbert]

@gauravtiwari Should I make a PR or should we close the issue?

[gauravtiwari]

@jeromedalbert Sorry about the delay, didn't fully understand the scope earlier.

I am not sure if we should support multiple entrypoint directories by default - for best practice reasons. Probably useful for app that is transitioning but ideally for developer experience it would be nice to have all entries at one place, which is how you would like to have in the longer term. You can also use subfolders for packs to namespace the entries a.k.a packs.

app/javascript:
  marketing: 
    src: 
      components: 
        - home.js 
        - calendar.js 

  calendar: 
    src: 
      components: 
        - chart.js 
  
  packs:
    marketing: 
      - home.js
      - calendar.js 

What do you think?

BTW made a PR (#557) for having multiple resolved_paths that way you can add the path to the array and then reference files from other folders to your packs directory.

resolved_paths: ['app/assets']

Then import them inside your packs like so:

// Note it's relative to parent directory i.e. app/assets
import 'stylesheets/main'
import 'images/rails.png'

As for the JS files, if I am not mistaken, it looks like the convention is to use app/javascript/packs/* files as references to app/javascript/* folders. For example app/javascript/packs/calendar.js's only content is import 'calendar', which loads app/javascript/calendar/index.js.

Taken from your above comment, no it's not just a reference. The packs are made solely for webpack entry points i.e. a file from where the compilation will start and will also emit a bundle of the same name. The app/javascript folder is where your source can live i.e. business logic or react components. All the bootstrapping i.e. mounting to DOM node can happen inside the entry file.