Writable Stream problem with end() callback

[tysenmoore-xse]

• Version: 5.1.1
• Platform: Linux bla-blah 3.13.0-86-generic lib: optimize require() path walking #130-Ubuntu SMP Mon Apr 18 18:27:15 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
• Subsystem: _stream_writable.js

I am trying to close a writable stream using end() with a callback. However, when end() calls endWritable() and state.finished is set to false so the callback is set to trigger when the finish event triggers. The problem is the state.finished flag is only set within finishMaybe(). This routine is only called from endWritable() and afterWrite(). However, once end() is called you cannot write more data to the stream to trigger afterWrite() to make state.finished = true. It would seem that the end() callback only works when the state.finished is true.

I compared the 5.1.1 version of the code with the current master head within git. There have really been no changes to this area of the code.

My question is, How do I get the end() callback to work? Is there a problem in the code? If so, is there a workaround? (so far I don't see one) Thanks for the time and any help.

[addaleax]

Could you provide e.g. a minimal snippet of code with an example where calling .end(…) does not work?

[tysenmoore-xse]

The code is rather large. To get a working example would be a bit of work, but basically the code is:

wavFile = fs.createReadStream( filename );
usbAudioWriter = getWritableStreamSomehow(); // calls a module that inherits from the Writable to stream over our USB API (The end code calls the Writeable.end as is.
wavFile.pipe( usbAudioWriter );
...
// When closing the stream down
usbAudioWriter.end(  function() {
     // Do the close here...
     console.log("End Called");    
} );

Like I mentioned before, when end() is called state.finished is false.

If I attempt to force the afterWrite() with something like the following code:
usbAudioWriter.write( Buffer("\0") );
I get the error: write after end

If this is not enough to go on, please let me know. I can attempt to reduce the problem to a testable use case. I am just working off of code inspection, but it seems if state.finished is false when endWritable() is called, the callback passed into end() will never be called--I could however be wrong. I am still rather new to all the streaming code.

[addaleax]

Ah! The docs for .pipe() might help clarify things. Normally, when piping from a stream to another, when the reading side ends, the writing sides ends, too, so you can’t call .end() again. If you want to know when writing to a stream ended, use the 'finish' event, i.e.

// When closing the stream down
usbAudioWriter.on( 'finish', function() {
     // Do the close here...
     console.log("End Called");    
} );

If there’s something you need to do before the writer stream ends, you should be able to override the default behaviour using {end: false} and ending the writable stream yourself:

wavFile = fs.createReadStream( filename );
usbAudioWriter = getWritableStreamSomehow(); // calls a module that inherits from the Writable to stream over our USB API (The end code calls the Writeable.end as is.
wavFile.pipe( usbAudioWriter, { end: false } );
...
// When closing the stream down
wavFile.on('finish', function() {
     // do things...
     usbAudioWriter.end(  function() {
          // Do the close here...
          console.log("End Called");    
     } );
} );

[tysenmoore-xse]

In my use case the wav file being played is on a continuous loop, so its end() is never called.

However, based on your suggestions I was able to get it working similar to your solution. Thanks so much for the time and help.

[akshita31]

I am working on a similar thing where I need to download a zip into to an already open file(that has a valid file descriptor) and also use this file descriptor later on to unzip the file. If I dont use pipe: false then the filestream end is coming and accessing the file descriptor for the unzip is giving an error because it is already closed. Is there a way to just know when the data has been flushed to the underlying stream without closing it ?