Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests less dependent on network resources #269

Open
mccalluc opened this issue Mar 7, 2025 · 0 comments
Open

Tests less dependent on network resources #269

mccalluc opened this issue Mar 7, 2025 · 0 comments
Labels
nice-to-have No discussion required, but not top priority

Comments

@mccalluc
Copy link
Contributor

mccalluc commented Mar 7, 2025

Ideally, unit tests should be able to run off line, so we're not surprised when some third party service goes off-line. I temporarily lost connectivity and was surprised by this error:

    def test_convert_nb_to_pdf():
        notebook = (fixtures_path / "fake-executed.ipynb").read_text()
>       pdf = convert_nb_to_pdf(notebook)

tests/utils/test_converters.py:81:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
dp_wizard/utils/converters.py:77: in convert_nb_to_pdf
    return convert_nb(python_nb, nbconvert.WebPDFExporter)
dp_wizard/utils/converters.py:95: in convert_nb
    (body, _resources) = exporter.from_notebook_node(notebook)
.venv-3.11/lib/python3.11/site-packages/nbconvert/exporters/webpdf.py:174: in from_notebook_node
    pdf_data = self.run_playwright(html)
.venv-3.11/lib/python3.11/site-packages/nbconvert/exporters/webpdf.py:163: in run_playwright
    pdf_data = pool.submit(run_coroutine, main(temp_file)).result()
/opt/homebrew/Cellar/[email protected]/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/concurrent/futures/_base.py:456: in result
    return self.__get_result()
/opt/homebrew/Cellar/[email protected]/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/concurrent/futures/_base.py:401: in __get_result
    raise self._exception
/opt/homebrew/Cellar/[email protected]/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/concurrent/futures/thread.py:58: in run
    result = self.fn(*self.args, **self.kwargs)
.venv-3.11/lib/python3.11/site-packages/nbconvert/exporters/webpdf.py:161: in run_coroutine
    return loop.run_until_complete(coro)
/opt/homebrew/Cellar/[email protected]/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py:653: in run_until_complete
    return future.result()
.venv-3.11/lib/python3.11/site-packages/nbconvert/exporters/webpdf.py:109: in main
    await page.goto(f"file://{temp_file.name}", wait_until="networkidle")
.venv-3.11/lib/python3.11/site-packages/playwright/async_api/_generated.py:8985: in goto
    await self._impl_obj.goto(
.venv-3.11/lib/python3.11/site-packages/playwright/_impl/_page.py:551: in goto
    return await self._main_frame.goto(**locals_to_params(locals()))
.venv-3.11/lib/python3.11/site-packages/playwright/_impl/_frame.py:145: in goto
    await self._channel.send("goto", locals_to_params(locals()))
.venv-3.11/lib/python3.11/site-packages/playwright/_impl/_connection.py:61: in send
    return await self._connection.wrap_api_call(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <playwright._impl._connection.Connection object at 0x1288a53d0>
cb = <function Channel.send.<locals>.<lambda> at 0x128ceb060>, is_internal = False

    async def wrap_api_call(
        self, cb: Callable[[], Any], is_internal: bool = False
    ) -> Any:
        if self._api_zone.get():
            return await cb()
        task = asyncio.current_task(self._loop)
        st: List[inspect.FrameInfo] = getattr(task, "__pw_stack__", inspect.stack())
        parsed_st = _extract_stack_trace_information_from_stack(st, is_internal)
        self._api_zone.set(parsed_st)
        try:
            return await cb()
        except Exception as error:
>           raise rewrite_error(error, f"{parsed_st['apiName']}: {error}") from None
E           playwright._impl._errors.TimeoutError: Page.goto: Timeout 30000ms exceeded.
E           Call log:
E             - navigating to "file:///var/folders/jm/40wztt291610f8pk5w0sw20r0000gq/T/tmp4gyfnnkg.html", waiting until "networkidle"

.venv-3.11/lib/python3.11/site-packages/playwright/_impl/_connection.py:528: TimeoutError

Apparently, playwright is used internally to render PDFs? and that code, in turn, depends on some other resource on the internet? Seems fragile, and if this tool could potentially be used with sensitive data, it would be really good to avoid random network connections!

@mccalluc mccalluc added the nice-to-have No discussion required, but not top priority label Mar 7, 2025
@github-project-automation github-project-automation bot moved this to Pending in DP Wizard Mar 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
nice-to-have No discussion required, but not top priority
Projects
Status: Pending
Development

No branches or pull requests

1 participant