Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reading of UTF8-encoded (w/ BOM) files fails #45

Closed
evil-shrike opened this issue May 13, 2016 · 2 comments
Closed

Reading of UTF8-encoded (w/ BOM) files fails #45

evil-shrike opened this issue May 13, 2016 · 2 comments

Comments

@evil-shrike
Copy link

This package would be much helper if it abstracts away utf8's BOM handling. Currently reading fails for files encoded in utf8 (JSON.parse throws "Unexpected symbol" at BOM).

Grunt's file module can solve this problem (see below). Why not to reimplement the similar approach here?

file.read = function(filepath, options) {
  if (!options) { options = {}; }
  var contents;
  grunt.verbose.write('Reading ' + filepath + '...');
  try {
    contents = fs.readFileSync(String(filepath));
    // If encoding is not explicitly null, convert from encoded buffer to a
    // string. If no encoding was specified, use the default.
    if (options.encoding !== null) {
      contents = iconv.decode(contents, options.encoding || file.defaultEncoding);
      // Strip any BOM that might exist.
      if (!file.preserveBOM && contents.charCodeAt(0) === 0xFEFF) {
        contents = contents.substring(1);
      }
    }
    grunt.verbose.ok();
    return contents;
  } catch(e) {
    grunt.verbose.error();
    throw grunt.util.error('Unable to read "' + filepath + '" file (Error code: ' + e.code + ').', e);
  }
};
@jprichardson jprichardson changed the title Reading of UTF8-encoded files fails Reading of UTF8-encoded (w/ BOM) files fails May 13, 2016
@jprichardson
Copy link
Owner

Hmm. It's recommended that in UTF8, there shouldn't be a BOM, but I realize that's not always the case. I'd accept a PR to strip the BOM. However, in writing a JSON file, it's not necessary.

@jprichardson
Copy link
Owner

Actually, I'll just fix it. Hang tight.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants