Skip to content
This repository was archived by the owner on Mar 5, 2025. It is now read-only.

Commit

Permalink
1.4.0 release
Browse files Browse the repository at this point in the history
  • Loading branch information
Lancetnik committed Jun 26, 2023
1 parent 87bb562 commit a06d370
Show file tree
Hide file tree
Showing 62 changed files with 724 additions and 175 deletions.
78 changes: 78 additions & 0 deletions docs/docs/en/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,67 @@
# CHANGELOG

## 2023-06-26 **0.1.4.0** PydanticV2

The main change in this update is the support for the **PydanticV2** beta version.

Also, this update **still supports Pydantic v1**, so if something with **PydanticV2** breaks you can simply roll it back - the latest **Propan** continues to work without changes.

Be careful: if you use **Propan** together with **FastAPI** when migrating to **PydanticV2**, you must install the version `fastapi>=0.100.0b1`, which is also compatible with both versions of **Pydantic**. However, if you are working on versions of **FastAPI** `0.9*`, the current release is compatible with them as well (but only using **PydanticV1**).

!!! quote ""
All test suites work correctly with all variations of the dependencies and on all supported Python versions.

Other changes:

Improved compatibility with **FastAPI**:

* **PropanRouter** supports top-level dependencies
```python
from propan.fastapi import RabbitRouter

router = RabbitRouter(dependencies=[...])
@router.event("test", dependencies=[...])
async def handler(a: str, b: int):
...
```

* You can test `router.event` using [`build_message`](../getting_started/7_testing/#regular-function-calling) directly
```python
import pytest, pydantic
from propan.fastapi import RabbitRouter
from propan.test.rabbit import build_message

router = RabbitRouter()

@router.event("test")
async def handler(a: str, b: int):
...

with pytest.raises(pydantic.ValidationError):
handler(build_message("Hello", "test"), reraise_exc=True)
```

Implemented [**BrokerRouter**](../getting_started/4_broker/2_routing/#brokerrouter) for the convenience of splitting the application code into imported submodules.

```python
from propan import RabbitBroker, RabbitRouter

router = RabbitRouter(prefix="user/")

@router.handle("created")
async def handle_user_created_event(user_id: str):
...

broker = RabbitBroker()
broker.include_router(router)
```

Added documentation [section](../getting_started/4_broker/4_custom_serialization/) about custom message serialization (using the example with *Protobuf*).

And also updated several other sections of the documentation, fixed several non-critical bugs, removed **RabbitBroker** *deprecated* methods, and increased test coverage of rare scenarios.

---

## 2023-06-14 **0.1.3.0** AsyncAPI

The current update adds functionality that I've been working hard on for the last month:
Expand All @@ -24,6 +86,8 @@ async def handler():
...
```

---

## 2023-06-13 **0.1.2.17**

The current update is a sum of several changes and improvements released from the previous release.
Expand Down Expand Up @@ -59,6 +123,8 @@ async def init_whatever(app: FastAPI): ...

In addition, the behavior of the `__init__` and `connect` methods for all brokers have been improved (now the `connect` parameters have priority and override the `__init__` parameters when connecting to the broker), a correct exception has been implemented when accessing an object unavailable for import, several errors have been fixed and other minor internal changes.

---

## 2023-05-28 **0.1.2.3** SQS Beta

**Propan** added support for *SQS* as a message broker. This functionality is full tested.
Expand All @@ -77,6 +143,8 @@ Also, current release include the following fixes:
* *Nats* connection recovery
* *Redis* connection methods supports not-url parameters

---

## 2023-05-26 **0.1.2.2** NATS Stable

`NatsBroker` is full tested now.
Expand All @@ -87,6 +155,8 @@ Also, to **Nats** supporting added:
* **RPC** supporting
* `NatsRouter` for **FastAPI**

---

## 2023-05-23 **0.1.2.0** Kafka

**Propan** added support for *Kafka* as a message broker. This functionality is full tested.
Expand All @@ -99,6 +169,8 @@ Also, to **Nats** supporting added:

*KafkaBroker* not supports **RPC** yet.

---

## 2023-05-18 **0.1.1.0** REDIS

**Propan** added support for *Redis Pub/Sub* as a message broker. This functionality is full tested and described in the documentation.
Expand All @@ -116,6 +188,8 @@ Also, **Propan CLI** is able to generate templates to any supported broker
propan create async [broker] [APPNAME]
```

---

## 2023-05-15 **0.1.0.0** STABLE

Stable and fully documented **Propan** release!
Expand All @@ -125,6 +199,8 @@ From the current release, no more breaking changes - use the framework safely!
At the current release, all *RabbitMQ* use cases are supported, tested, and described in the documentation.
Expect support for *Redis* (testing now), *Kafka* (in development), and full support for *Nats* (also in development) soon.

---

## 2023-05-01 **0.0.9.4**

Great news! Now **Propan** can be used as a full part of **FastAPI**!
Expand Down Expand Up @@ -172,6 +248,8 @@ Also added support for [RPC over MQ](../getting_started/4_broker/5_rpc) (RabbitM
* Brokers `publish_message` method has been renamed to `publish`
* removed `declare` argument in `RabbitQueue` and `RabbitExchange` - now you need to use `passive`

---

## 2023-04-18 **0.0.9**

Release is timed to accompany the release of [fast-depends](https://lancetnik.github.io/FastDepends/).
Expand Down
17 changes: 16 additions & 1 deletion docs/docs/en/getting_started/4_broker/2_routing.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,24 @@ To learn more about the behavior of specialized brokers, go to the following sec
* [NatsBroker](../../../nats/1_nats-index/#routing-rules)
* [RedisBroker](../../../redis/1_redis-index/#routing-rules)

## BrokerRouter

Sometimes it can be convenient to divide handlers into groups that can be connected to your application with just one command.
To do this, **Propan** provides **BrokerRouter**: you can register your handlers within the router, and then connect this router to your broker.

This will help to better organize your application's code and also allow you to divide it into plug-ins.

{! includes/getting_started/broker/routers.md !}

In this case, the router prefix will be added to the name of the queue of your handlers.

```python hl_lines="3"
{!> docs_src/quickstart/broker/routers/publish.py !}
```

## Error handling

However, all brokers have the `retry` flag in the `@broker.hanle` method, which is responsible for error handling logic.
However, all brokers supporting acknowledgement have the `retry` flag in the `@broker.hanle` method, which is responsible for error handling logic.

By default, this flag has the value `False`, which indicates that if an error occurred during message processing, it will still be retrieved from the queue.

Expand Down
88 changes: 88 additions & 0 deletions docs/docs/en/getting_started/4_broker/4_custom_serialization.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
# Custom Serialization

By default, **Propan** uses the *JSON* format to send and receive messages. However, if you need to process messages in other formats or with additional steps for serialization (*gzip*, *Avro*, *Protobuf*, etc.), you can modify the serialization logic.

## Serialization Steps

Before the message gets into your handler, **Propan** applies 2 functions to it sequentially: `parse_message` and `decode_message`. You can modify one or both stages depending on your needs.

### Message Parsing

At this stage, **Propan** serializes an incoming message of the framework that is used to work with the broker into a general view - **PropanMessage**. At this stage, the message body remains in the form of raw bytes.

The signature of the function looks like this:

{! includes/getting_started/broker/serialization/parse_signature.md !}

This stage is strongly related to the features of the broker used and in most cases, its redefinition is not necessary.

However, it is still possible. You can override this method both for the entire broker and for individual handlers:

```python linenums="1" hl_lines="3 6 8"
{!> docs_src/quickstart/broker/serialization/2_parse_redefine.py !}
```

Your function should take 2 arguments: a "raw" message itself and the original handler function. Thus, you can either completely redefine the message parsing logic, or partially modify a message, and then use the original **Propan** mechanism.

The parser declared at the `broker` level will be applied to all handlers. The parser declared at the `handle` level is applied only to this handler (it ignores the `broker' parser if it was specified earlier).

### Message Decoding

At this stage, the body of **PropanMessage** is transformed to the form in which it enters your handler function. This method you will have to redefine more often.

In the original, its signature is quite simple (this is a simplified version):

```python linenums="1"
{!> docs_src/quickstart/broker/serialization/3_decode.py !}
```

To redefine it, use the same way as the parser:

```python linenums="1" hl_lines="3 6 8"
{!> docs_src/quickstart/broker/serialization/4_decode_redefine.py !}
```

## Example with Protobuf

In this section, we will look at an example using *Protobuf*, however, it is also applicable for any other serialization methods.

???- note "Protobuf"
*Protobuf* is an alternative message serialization method commonly used in *GRPC*. Its main advantage is much smaller [^1] message size (compared to *JSON*), but it requires a message schema (`.proto` files) both on the client side and on the server side.

To begin with, install the dependencies:

```console
pip install grpcio-tools
```

Then we will describe the scheme of our message

```proto title="message.proto"
syntax = "proto3";
message Person {
string name = 1;
float age = 2;
}
```

Now we will generate a *Python* class for working with messages in the *Protobuf format*

```console
python -m grpc_tools.protoc --python_out=. --pyi_out=. -I . message.proto
```

At the output, we get 2 files: `message_pb2.py` and `message_pb2.pyi`. Now we are ready to use the generated class to serialize our messages.

```python linenums="1" hl_lines="1 10-13 15 21"
{!> docs_src/quickstart/broker/serialization/5_protobuf.py !}
```

Note that we used the `NoCast` annotation, which excludes the message from the `pydantic` representation of our handler.

```python
{!> docs_src/quickstart/broker/serialization/5_protobuf.py [ln:16] !}
```

[^1]:
For example, a message like `{ "name": "john", "age": 25 }` in *JSON* takes **27** bytes, and in *Protobuf* - **11**. With lists and more complex structures, the savings can be even more significant (up to 20x times).
Original file line number Diff line number Diff line change
Expand Up @@ -45,4 +45,4 @@ broker as context manager to send.
Within this context, you can send an unlimited number of messages, as well as synchronously wait for a response to them.
However, `handle` cannot be initialized within this context: they will complete their execution along with the context.

This will be discribed in more detail in the next section.
This will be discribed in more detail in the next section.
21 changes: 10 additions & 11 deletions docs/docs/en/getting_started/7_testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,21 +19,18 @@ Let's image we have an application like so:
{% import 'getting_started/test/1.md' as includes with context %}
{{ includes }}

To test handlers at first we should run the broker:
Then make an *RPC* request to check the result of the execution:

{! includes/getting_started/test/2.md !}

Then make an *RPC* request to check the result of the execution:

{! includes/getting_started/test/3.md !}
!!! note
Using test broker this way **RPC** is always able, even broker doesn't support it in regular mode.

## Using fixtures

For large applications to reuse the test broker, you can use the following fixture:

{! includes/getting_started/test/4.md !}

## Regular function calling
{! includes/getting_started/test/3.md !}

!!! tip
This approach has a major weakness: Errors that raises inside handler **cannot be captured** inside your tests.
Expand All @@ -46,14 +43,16 @@ For example, the following test will return `None` and inside the handler, a `py

Also this test will be blocked for `callback_timeout` (default **30** seconds), which can be very annoying when a handler development error occures, and your tests fail with a long timeout of `None`.

Therefore, **Propan** provides the ability to run handler functions as if they were regular functions.
## Regular function calling

To do this, you need to construct a message using the `build_message`, if it was `pubslih` (same method signatures), and passe this message to your handler as the single function argument.
**Propan** provides the ability to run handler functions as if they were regular functions.

{! includes/getting_started/test/5.md !}
To do this, you need to construct a message using the `build_message`, if it was `publish` (same method signatures), and passe this message to your handler as the single function argument.

{! includes/getting_started/test/4.md !}

That being said, if you want to catch handler exceptions, you need to use the `reraise_exc=True` calling flag:

{! includes/getting_started/test/6.md !}
{! includes/getting_started/test/5.md !}

Thus, **Propan** provides you with a complete toolkit for testing your handlers, from checking *RPC* responses to correctly executing body functions.
Loading

0 comments on commit a06d370

Please sign in to comment.