Merge pull request #529 from google/google_sync

Google sync

README.md

@@ -60,7 +60,7 @@ Also see the corresponding [FAQ entry][faq-diff].
 To quickly get started with type-checking a file or directory, run the
 following, replacing `file_or_directory` with your input:
 
-```
+```shell
 pip install pytype
 pytype file_or_directory
 ```
@@ -82,7 +82,7 @@ Finally, pytype generates files of inferred type information, located by default
 in `.pytype/pyi`. You can use this information to type-annotate the
 corresponding source file:
 
-```
+```shell
 merge-pyi -i <filepath>.py .pytype/pyi/<filename>.pyi
 ```
 
@@ -104,26 +104,31 @@ Pytype can be installed via pip. Note that the installation requires `wheel`
 and `setuptools`. (If you're working in a virtualenv, these two packages should
 already be present.)
 
-```
+```shell
 pip install pytype
 ```
 
 Or from the source code [on GitHub][github].
 
-```
+```shell
 git clone --recurse-submodules https://github.com/google/pytype.git
 cd pytype
-pip install -U .
+pip install .
 ```
 
 Instead of using `--recurse-submodules`, you could also have run
 
-```
+```shell
 git submodule init
 git submodule update
 ```
 
-in the `pytype` directory.
+in the `pytype` directory. To edit the code and have your edits tracked live,
+replace the pip install command with:
+
+```shell
+pip install -e .
+```
 
 ### Installing on WSL
 
@@ -166,7 +171,7 @@ is not supplied, pytype will look for a `[pytype]` section in the first
 
 Start off by generating a sample config file:
 
-```
+```shell
 $ pytype --generate-config pytype.cfg
 ```
 

docs/dev_guide.md

@@ -1 +1,151 @@
+# Developer guide
+
 **Under Construction**
+
+<!--ts-->
+   * [Developer guide](#developer-guide)
+      * [Development process](#development-process)
+         * [GitHub](#github)
+         * [Running pytype locally](#running-pytype-locally)
+         * [Debugging](#debugging)
+         * [Profiling](#profiling)
+      * [Introduction](#introduction)
+      * [Basic concepts](#basic-concepts)
+      * [Adding a new feature](#adding-a-new-feature)
+      * [Important code components](#important-code-components)
+         * [AST representation of type stubs](#ast-representation-of-type-stubs)
+            * [Where stubs are located](#where-stubs-are-located)
+         * [Abstract representation of types](#abstract-representation-of-types)
+         * [Conversion between representations](#conversion-between-representations)
+         * [Bytecode handling](#bytecode-handling)
+         * [CFG](#cfg)
+
+<!-- Added by: rechen, at: 2020-03-10T14:26-07:00 -->
+
+<!--te-->
+
+## Development process
+
+### GitHub
+
+1. Fork https://github.com/google/pytype.
+
+1. Follow the [instructions][source-install-instructions] for installing from
+   source, using your fork instead of the original repository. Now the `pytype`
+   and `pytype-single` command-line tools are running from your local copy of
+   the pytype source code. Make sure to use `pip install -e .` so that the tools
+   will automatically pick up code edits.
+
+1. Make your change, adding [tests][tests-readme-oss] as appropriate.
+
+1. Make sure the code still passes all [tests][tests-readme-oss] and is free of
+   [lint][pylint] and [type][pytype-quickstart] errors.
+
+1. Push your change to your fork and open a PR against the original repo. If
+   it's your first time contributing to a Google open source project, please
+   sign the Contributor License Agreement when prompted. Depending on what files
+   your PR touches, it will be either merged directly or closed after being
+   copied into the Google-internal codebase and re-exported to GitHub. You will
+   be credited as the author either way.
+
+### Running pytype locally
+
+Run the single-file analysis tool as
+
+```shell
+pytype-single some_file.py
+```
+
+to check `some_file.py` for type errors, or
+
+```shell
+pytype-single some_file.py -o -
+```
+
+to infer a pyi (dumped to stdout via `-`). The default target Python
+version is the version that pytype is running under; pass in `-V<major>.<minor>`
+to select a different version.
+
+Note that the single-file tool does not handle dependency resolution, so
+you'll have to supply .pyi files for all non-stdlib imports.
+
+If you're using the GitHub-installed tools, you can run the whole-project
+analysis tool, `pytype`, over the file to generate a `.pytype` directory that
+includes the necessary .pyi files. Then add
+
+```shell
+--module-name <module> --imports_info .pytype/imports/<module>.imports
+```
+
+to your `pytype-single` invocation, replacing `<module>` with the fully
+qualified module name.
+
+### Debugging
+
+Use the `--verbosity` (or `-v`) option to print debugging output. The possible
+values range from -1 ("quiet", log nothing) to 4 ("debug", log everything).
+
+pytype can be run under [pdb][pdb], the Python debugger. Add:
+
+```python
+import pdb; pdb.set_trace()
+```
+
+at the point in the code at which you want to enter debugging.
+
+### Profiling
+
+For profiling a single file, pass in `--profile <path>` to turn on profiling and
+write the generated profile to `<path>`. The profile can then be analyzed using
+the [`pstats`][pstats] module.
+
+Note that numbers are not directly comparable between runs; differences of 100%
+for different machines on otherwise identical code have happened. The relative
+rank of functions in the profile is stable between runs.
+
+## Introduction
+
+How to trace pytype, intro to bytecode interpreter
+
+## Basic concepts
+
+What's a variable, etc.
+
+## Adding a new feature
+
+pytd node, abstract value, conversion both ways
+
+## Important code components
+
+### AST representation of type stubs
+
+`pyi/` (parser), `pytd/`
+
+#### Where stubs are located
+
+typeshed, `pytd/builtins/`, `pytd/stdlib/`,
+`//devtools/python/blaze/pytype/overrides/`
+
+### Abstract representation of types
+
+`abstract.py`, `matcher.py`, `overlays/`
+
+### Conversion between representations
+
+`convert.py`, `output.py`
+
+### Bytecode handling
+
+`pyc/`, `vm.py`
+
+### CFG
+
+`typegraph/`
+
+<!-- General references -->
+[pdb]: https://docs.python.org/3/library/pdb.html
+[pylint]: http://pylint.pycqa.org/en/latest/
+[pytype-quickstart]: https://github.com/google/pytype#quickstart
+[pstats]: https://docs.python.org/3/library/profile.html#module-pstats
+[source-install-instructions]: https://github.com/google/pytype#installing
+[tests-readme-oss]: https://github.com/google/pytype/blob/master/pytype/tests/README.md

docs/index.md

@@ -58,7 +58,7 @@ Also see the corresponding [FAQ entry][faq-diff].
 To quickly get started with type-checking a file or directory, run the
 following, replacing `file_or_directory` with your input:
 
-```
+```shell
 pip install pytype
 pytype file_or_directory
 ```
@@ -80,7 +80,7 @@ Finally, pytype generates files of inferred type information, located by default
 in `.pytype/pyi`. You can use this information to type-annotate the
 corresponding source file:
 
-```
+```shell
 merge-pyi -i <filepath>.py .pytype/pyi/<filename>.pyi
 ```
 
@@ -102,26 +102,31 @@ Pytype can be installed via pip. Note that the installation requires `wheel`
 and `setuptools`. (If you're working in a virtualenv, these two packages should
 already be present.)
 
-```
+```shell
 pip install pytype
 ```
 
 Or from the source code [on GitHub][github].
 
-```
+```shell
 git clone --recurse-submodules https://github.com/google/pytype.git
 cd pytype
-pip install -U .
+pip install .
 ```
 
 Instead of using `--recurse-submodules`, you could also have run
 
-```
+```shell
 git submodule init
 git submodule update
 ```
 
-in the `pytype` directory.
+in the `pytype` directory. To edit the code and have your edits tracked live,
+replace the pip install command with:
+
+```shell
+pip install -e .
+```
 
 ### Installing on WSL
 
@@ -164,7 +169,7 @@ is not supplied, pytype will look for a `[pytype]` section in the first
 
 Start off by generating a sample config file:
 
-```
+```shell
 $ pytype --generate-config pytype.cfg
 ```
 

pytype/pytd/builtins/2/typing.pytd

@@ -236,7 +236,7 @@ class IO(Iterator[AnyStr]):
   def tell(self) -> int: ...
   def truncate(self, size: int = ...) -> Optional[int]: ...
   def writable(self) -> bool: ...
-  def write(self, s: Union[bytes, Text, bytearray]) -> int: ...
+  def write(self, s: AnyStr) -> int: ...
   def writelines(self, lines: Iterable[AnyStr]) -> None: ...
 
 

pytype/pytd/builtins/2and3/typing.pytd

@@ -236,7 +236,7 @@ class IO(Iterator[AnyStr]):
   def tell(self) -> int: ...
   def truncate(self, size: int = ...) -> Optional[int]: ...
   def writable(self) -> bool: ...
-  def write(self, s: Union[bytes, Text, bytearray]) -> int: ...
+  def write(self, s: AnyStr) -> int: ...
   def writelines(self, lines: Iterable[AnyStr]) -> None: ...
 
 

pytype/pytd/builtins/3/typing.pytd

@@ -236,7 +236,7 @@ class IO(Iterator[AnyStr]):
   def tell(self) -> int: ...
   def truncate(self, size: int = ...) -> Optional[int]: ...
   def writable(self) -> bool: ...
-  def write(self, s: Union[bytes, Text, bytearray]) -> int: ...
+  def write(self, s: AnyStr) -> int: ...
   def writelines(self, lines: Iterable[AnyStr]) -> None: ...
 
 

pytype/tests/test_builtins3.py

@@ -157,7 +157,7 @@ def testInitWithUnicode(self):
   def testIOWrite(self):
     self.Check("""
         import sys
-        sys.stdout.write(bytearray([1,2,3]))
+        sys.stdout.write("hello world")
     """)
 
   def testHasAttrNone(self):