diff --git a/README.md b/README.md index 9f76eb362..f5287840a 100755 --- a/README.md +++ b/README.md @@ -60,7 +60,7 @@ Also see the corresponding [FAQ entry][faq-diff]. To quickly get started with type-checking a file or directory, run the following, replacing `file_or_directory` with your input: -``` +```shell pip install pytype pytype file_or_directory ``` @@ -82,7 +82,7 @@ Finally, pytype generates files of inferred type information, located by default in `.pytype/pyi`. You can use this information to type-annotate the corresponding source file: -``` +```shell merge-pyi -i .py .pytype/pyi/.pyi ``` @@ -104,26 +104,31 @@ Pytype can be installed via pip. Note that the installation requires `wheel` and `setuptools`. (If you're working in a virtualenv, these two packages should already be present.) -``` +```shell pip install pytype ``` Or from the source code [on GitHub][github]. -``` +```shell git clone --recurse-submodules https://github.com/google/pytype.git cd pytype -pip install -U . +pip install . ``` Instead of using `--recurse-submodules`, you could also have run -``` +```shell git submodule init git submodule update ``` -in the `pytype` directory. +in the `pytype` directory. To edit the code and have your edits tracked live, +replace the pip install command with: + +```shell +pip install -e . +``` ### Installing on WSL @@ -166,7 +171,7 @@ is not supplied, pytype will look for a `[pytype]` section in the first Start off by generating a sample config file: -``` +```shell $ pytype --generate-config pytype.cfg ``` diff --git a/docs/dev_guide.md b/docs/dev_guide.md index 765babe38..22b06bcd9 100644 --- a/docs/dev_guide.md +++ b/docs/dev_guide.md @@ -1 +1,151 @@ +# Developer guide + **Under Construction** + + + * [Developer guide](#developer-guide) + * [Development process](#development-process) + * [GitHub](#github) + * [Running pytype locally](#running-pytype-locally) + * [Debugging](#debugging) + * [Profiling](#profiling) + * [Introduction](#introduction) + * [Basic concepts](#basic-concepts) + * [Adding a new feature](#adding-a-new-feature) + * [Important code components](#important-code-components) + * [AST representation of type stubs](#ast-representation-of-type-stubs) + * [Where stubs are located](#where-stubs-are-located) + * [Abstract representation of types](#abstract-representation-of-types) + * [Conversion between representations](#conversion-between-representations) + * [Bytecode handling](#bytecode-handling) + * [CFG](#cfg) + + + + + +## Development process + +### GitHub + +1. Fork https://github.com/google/pytype. + +1. Follow the [instructions][source-install-instructions] for installing from + source, using your fork instead of the original repository. Now the `pytype` + and `pytype-single` command-line tools are running from your local copy of + the pytype source code. Make sure to use `pip install -e .` so that the tools + will automatically pick up code edits. + +1. Make your change, adding [tests][tests-readme-oss] as appropriate. + +1. Make sure the code still passes all [tests][tests-readme-oss] and is free of + [lint][pylint] and [type][pytype-quickstart] errors. + +1. Push your change to your fork and open a PR against the original repo. If + it's your first time contributing to a Google open source project, please + sign the Contributor License Agreement when prompted. Depending on what files + your PR touches, it will be either merged directly or closed after being + copied into the Google-internal codebase and re-exported to GitHub. You will + be credited as the author either way. + +### Running pytype locally + +Run the single-file analysis tool as + +```shell +pytype-single some_file.py +``` + +to check `some_file.py` for type errors, or + +```shell +pytype-single some_file.py -o - +``` + +to infer a pyi (dumped to stdout via `-`). The default target Python +version is the version that pytype is running under; pass in `-V.` +to select a different version. + +Note that the single-file tool does not handle dependency resolution, so +you'll have to supply .pyi files for all non-stdlib imports. + +If you're using the GitHub-installed tools, you can run the whole-project +analysis tool, `pytype`, over the file to generate a `.pytype` directory that +includes the necessary .pyi files. Then add + +```shell +--module-name --imports_info .pytype/imports/.imports +``` + +to your `pytype-single` invocation, replacing `` with the fully +qualified module name. + +### Debugging + +Use the `--verbosity` (or `-v`) option to print debugging output. The possible +values range from -1 ("quiet", log nothing) to 4 ("debug", log everything). + +pytype can be run under [pdb][pdb], the Python debugger. Add: + +```python +import pdb; pdb.set_trace() +``` + +at the point in the code at which you want to enter debugging. + +### Profiling + +For profiling a single file, pass in `--profile ` to turn on profiling and +write the generated profile to ``. The profile can then be analyzed using +the [`pstats`][pstats] module. + +Note that numbers are not directly comparable between runs; differences of 100% +for different machines on otherwise identical code have happened. The relative +rank of functions in the profile is stable between runs. + +## Introduction + +How to trace pytype, intro to bytecode interpreter + +## Basic concepts + +What's a variable, etc. + +## Adding a new feature + +pytd node, abstract value, conversion both ways + +## Important code components + +### AST representation of type stubs + +`pyi/` (parser), `pytd/` + +#### Where stubs are located + +typeshed, `pytd/builtins/`, `pytd/stdlib/`, +`//devtools/python/blaze/pytype/overrides/` + +### Abstract representation of types + +`abstract.py`, `matcher.py`, `overlays/` + +### Conversion between representations + +`convert.py`, `output.py` + +### Bytecode handling + +`pyc/`, `vm.py` + +### CFG + +`typegraph/` + + +[pdb]: https://docs.python.org/3/library/pdb.html +[pylint]: http://pylint.pycqa.org/en/latest/ +[pytype-quickstart]: https://github.com/google/pytype#quickstart +[pstats]: https://docs.python.org/3/library/profile.html#module-pstats +[source-install-instructions]: https://github.com/google/pytype#installing +[tests-readme-oss]: https://github.com/google/pytype/blob/master/pytype/tests/README.md diff --git a/docs/index.md b/docs/index.md index 2e173e737..1d8e58fb3 100755 --- a/docs/index.md +++ b/docs/index.md @@ -58,7 +58,7 @@ Also see the corresponding [FAQ entry][faq-diff]. To quickly get started with type-checking a file or directory, run the following, replacing `file_or_directory` with your input: -``` +```shell pip install pytype pytype file_or_directory ``` @@ -80,7 +80,7 @@ Finally, pytype generates files of inferred type information, located by default in `.pytype/pyi`. You can use this information to type-annotate the corresponding source file: -``` +```shell merge-pyi -i .py .pytype/pyi/.pyi ``` @@ -102,26 +102,31 @@ Pytype can be installed via pip. Note that the installation requires `wheel` and `setuptools`. (If you're working in a virtualenv, these two packages should already be present.) -``` +```shell pip install pytype ``` Or from the source code [on GitHub][github]. -``` +```shell git clone --recurse-submodules https://github.com/google/pytype.git cd pytype -pip install -U . +pip install . ``` Instead of using `--recurse-submodules`, you could also have run -``` +```shell git submodule init git submodule update ``` -in the `pytype` directory. +in the `pytype` directory. To edit the code and have your edits tracked live, +replace the pip install command with: + +```shell +pip install -e . +``` ### Installing on WSL @@ -164,7 +169,7 @@ is not supplied, pytype will look for a `[pytype]` section in the first Start off by generating a sample config file: -``` +```shell $ pytype --generate-config pytype.cfg ``` diff --git a/pytype/pytd/builtins/2/typing.pytd b/pytype/pytd/builtins/2/typing.pytd index 1dd0655c5..1bf6af9f4 100644 --- a/pytype/pytd/builtins/2/typing.pytd +++ b/pytype/pytd/builtins/2/typing.pytd @@ -236,7 +236,7 @@ class IO(Iterator[AnyStr]): def tell(self) -> int: ... def truncate(self, size: int = ...) -> Optional[int]: ... def writable(self) -> bool: ... - def write(self, s: Union[bytes, Text, bytearray]) -> int: ... + def write(self, s: AnyStr) -> int: ... def writelines(self, lines: Iterable[AnyStr]) -> None: ... diff --git a/pytype/pytd/builtins/2and3/typing.pytd b/pytype/pytd/builtins/2and3/typing.pytd index 1dd0655c5..1bf6af9f4 100644 --- a/pytype/pytd/builtins/2and3/typing.pytd +++ b/pytype/pytd/builtins/2and3/typing.pytd @@ -236,7 +236,7 @@ class IO(Iterator[AnyStr]): def tell(self) -> int: ... def truncate(self, size: int = ...) -> Optional[int]: ... def writable(self) -> bool: ... - def write(self, s: Union[bytes, Text, bytearray]) -> int: ... + def write(self, s: AnyStr) -> int: ... def writelines(self, lines: Iterable[AnyStr]) -> None: ... diff --git a/pytype/pytd/builtins/3/typing.pytd b/pytype/pytd/builtins/3/typing.pytd index 1dd0655c5..1bf6af9f4 100644 --- a/pytype/pytd/builtins/3/typing.pytd +++ b/pytype/pytd/builtins/3/typing.pytd @@ -236,7 +236,7 @@ class IO(Iterator[AnyStr]): def tell(self) -> int: ... def truncate(self, size: int = ...) -> Optional[int]: ... def writable(self) -> bool: ... - def write(self, s: Union[bytes, Text, bytearray]) -> int: ... + def write(self, s: AnyStr) -> int: ... def writelines(self, lines: Iterable[AnyStr]) -> None: ... diff --git a/pytype/tests/test_builtins3.py b/pytype/tests/test_builtins3.py index a88693791..f0919f3b1 100644 --- a/pytype/tests/test_builtins3.py +++ b/pytype/tests/test_builtins3.py @@ -157,7 +157,7 @@ def testInitWithUnicode(self): def testIOWrite(self): self.Check(""" import sys - sys.stdout.write(bytearray([1,2,3])) + sys.stdout.write("hello world") """) def testHasAttrNone(self):