Merge: (plz) Add :filter argument for process filter

README.org

@@ -142,6 +142,9 @@ Synchronously download a JPEG file, then create an Emacs image object from the d
 
    ~NOQUERY~ is passed to ~make-process~, which see.
 
+   ~FILTER~ is an optional function to be used as the process filter for the curl process.  It can be used to handle HTTP responses in a streaming way.  The function must accept 2 arguments, the process object running curl, and a string which is output received from the process.  The default process filter inserts the output of the process into the process buffer.  The provided ~FILTER~ function should at least insert output up to the HTTP body into the process buffer.
+
+
 ** Queueing
 
 ~plz~ provides a simple system for queueing HTTP requests.  First, make a ~plz-queue~ struct by calling ~make-plz-queue~.  Then call ~plz-queue~ with the struct as the first argument, and the rest of the arguments being the same as those passed to ~plz~.  Then call ~plz-run~ to run the queued requests.
@@ -187,7 +190,9 @@ You may also clear a queue with ~plz-clear~, which cancels any active or queued
 
 ** 0.8-pre
 
-Nothing new yet.
+*Additions*
+
++ Function ~plz~ now accepts a ~:filter~ argument which can be used to override the default process filter (e.g. for streaming responses).  ([[https://github.com/alphapapa/plz.el/pull/43][#43]], [[https://github.com/alphapapa/plz.el/pull/50][#50]].  Thanks to [[https://github.com/r0man][Roman Scherer]].)
 
 ** 0.7.3
 

plz.el

@@ -254,7 +254,7 @@ connection phase and waiting to receive the response (the
 
 ;;;;; Public
 
-(cl-defun plz (method url &rest rest &key headers body else finally noquery
+(cl-defun plz (method url &rest rest &key headers body else filter finally noquery
                       (as 'string) (then 'sync)
                       (body-type 'text) (decode t decode-s)
                       (connect-timeout plz-connect-timeout) (timeout plz-timeout))
@@ -330,6 +330,15 @@ from a host, respectively.
 
 NOQUERY is passed to `make-process', which see.
 
+FILTER is an optional function to be used as the process filter
+for the curl process.  It can be used to handle HTTP responses in
+a streaming way.  The function must accept 2 arguments, the
+process object running curl, and a string which is output
+received from the process.  The default process filter inserts
+the output of the process into the process buffer.  The provided
+FILTER function should at least insert output up to the HTTP body
+into the process buffer.
+
 \(To silence checkdoc, we mention the internal argument REST.)"
   ;; FIXME(v0.8): Remove the note about error changes from the docstring.
   ;; FIXME(v0.8): Update error signals in docstring.
@@ -404,6 +413,7 @@ NOQUERY is passed to `make-process', which see.
                                 :coding 'binary
                                 :command (append (list plz-curl-program) curl-command-line-args)
                                 :connection-type 'pipe
+                                :filter filter
                                 :sentinel #'plz--sentinel
                                 :stderr stderr-process
                                 :noquery noquery))

tests/test-plz.el

@@ -559,6 +559,49 @@ and only called once."
 
 ;; TODO: Add test for canceling queue.
 
+;; Process filter
+
+(defun test-plz-process-filter (process output)
+  "Write OUTPUT to the PROCESS buffer."
+  (when (buffer-live-p (process-buffer process))
+    (with-current-buffer (process-buffer process)
+      (let ((moving (= (point) (process-mark process))))
+        (save-excursion
+          (goto-char (process-mark process))
+          (insert output)
+          (set-marker (process-mark process) (point)))
+        (if moving (goto-char (process-mark process)))))))
+
+(plz-deftest plz-get-json-process-filter-async ()
+  (let* ((test-json) (outputs)
+         (process (plz 'get (url "/get")
+                    :as #'json-read
+                    :then (lambda (json)
+                            (setf test-json json))
+                    :filter (lambda (process output)
+                              (test-plz-process-filter process output)
+                              (push output outputs)))))
+    (plz-test-wait process)
+    (let-alist test-json
+      (should (string-match "curl" .headers.User-Agent)))
+    (let ((output (string-join (reverse outputs))))
+      (should (string-match "HTTP.*\s+200" output))
+      (should (string-match "Server: gunicorn" output))
+      (should (string-match "\"args\":\s*{}" output)))))
+
+(plz-deftest plz-get-json-process-filter-sync ()
+  (let* ((outputs)
+         (response (plz 'get (url "/get")
+                    :as 'response
+                    :filter (lambda (process output)
+                              (test-plz-process-filter process output)
+                              (push output outputs)))))
+    (plz-test-get-response response)
+    (let ((output (string-join (reverse outputs))))
+      (should (string-match "HTTP.*\s+200" output))
+      (should (string-match "Server: gunicorn" output))
+      (should (string-match "\"args\":\s*{}" output)))))
+
 ;;;; Footer
 
 (provide 'test-plz)