Skip to content

Commit 7fd66ce

Browse files
committed
Merge branch 'main' into rds/add-course-name-fields
2 parents 82828a6 + 3cef2d7 commit 7fd66ce

File tree

80 files changed

+5227
-3483
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

80 files changed

+5227
-3483
lines changed

.github/workflows/test-tutor-aspects.yml

+3-3
Original file line numberDiff line numberDiff line change
@@ -97,18 +97,18 @@ jobs:
9797
- name: Tutor build openedx
9898
run: tutor images build openedx-dev aspects aspects-superset
9999
- name: Free Disk Space (Ubuntu)
100-
uses: jlumbroso/free-disk-space@main
100+
uses: jlumbroso/free-disk-space@4d9e71b726748f254fe64fa44d273194bd18ec91
101101
with:
102102
# this might remove tools that are actually needed,
103103
# if set to "true" but frees about 6 GB
104104
tool-cache: false
105-
105+
106106
# all of these default to true, but feel free to set to
107107
# "false" if necessary for your workflow
108108
android: true
109109
dotnet: true
110110
haskell: true
111-
large-packages: true
111+
large-packages: false
112112
swap-storage: true
113113
- name: Create k8s Kind Cluster
114114
uses: helm/[email protected]

CHANGELOG.md

+30
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,36 @@ All notable changes to this project will be documented in this file.
55
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
66
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
77

8+
## v0.26.0 - 2023-08-15
9+
10+
### [0.26.0](https://github.com/openedx/tutor-contrib-aspects/compare/v0.25.1...v0.26.0) (2023-08-15)
11+
12+
#### Features
13+
14+
- Update Operator Dashboard ([3659f90](https://github.com/openedx/tutor-contrib-aspects/commit/3659f904768e9233c4ed95f9a676dff486f3270d))
15+
16+
#### Bug Fixes
17+
18+
- Re-add roles to dashboards, update docs, update import script to force roles ([bcab2a0](https://github.com/openedx/tutor-contrib-aspects/commit/bcab2a01a40094f77ae9bd41a5ed3e57765c3561))
19+
- Remove unnecessary date filter from unique actors chart ([598b794](https://github.com/openedx/tutor-contrib-aspects/commit/598b794332221b35055d1a36a4d9309ac0c35b00))
20+
21+
#### Documentation
22+
23+
- Update asset contribution readme section ([12fb7bf](https://github.com/openedx/tutor-contrib-aspects/commit/12fb7bfead8fec3c1772248fd6ab107789bddafe))
24+
25+
#### Build Systems
26+
27+
- Pin disk space cleaner, try to fix build error ([5b1fef2](https://github.com/openedx/tutor-contrib-aspects/commit/5b1fef207900332f516237657f585516bec3fbe0))
28+
29+
## v0.25.1 - 2023-08-14
30+
31+
### [0.25.1](https://github.com/openedx/tutor-contrib-aspects/compare/v0.25.0...v0.25.1) (2023-08-14)
32+
33+
### Bug Fixes
34+
35+
- Adapted transform_tracking_logs command for enhanced CLI integration. ([dfcf820](https://github.com/openedx/tutor-contrib-aspects/commit/dfcf8208ce181150575899123a51d54aa3ed7bac))
36+
- add configurable clickhouse volume size ([e513d02](https://github.com/openedx/tutor-contrib-aspects/commit/e513d02648b314fd7ed1ced934609d2e4c0f260d))
37+
838
## v0.25.0 - 2023-08-08
939

1040
### [0.25.0](https://github.com/openedx/tutor-contrib-aspects/compare/v0.24.0...v0.25.0) (2023-08-08)

LICENSE

+661
Large diffs are not rendered by default.

README.rst

+38-16
Original file line numberDiff line numberDiff line change
@@ -93,10 +93,11 @@ Sharing Charts and Dashboards
9393
To share your charts with others in the community, use Superset's "Export" button to
9494
save a zip file of your charts and related datasets.
9595

96-
.. note::
96+
.. warning::
9797
The exported datasets will contain hard-coded references to your particular
98-
databases, including your database hostname, port, and username, but it
99-
will not contain passwords.
98+
databases, including your database hostname, port, and username, in some cases
99+
it may also contain database passwords. It is vital that you review the
100+
database and dataset files before sharing them.
100101

101102
To import charts or dashboards shared by someone in the community:
102103

@@ -109,29 +110,50 @@ To import charts or dashboards shared by someone in the community:
109110
Contributing Charts and Dashboards to Aspects
110111
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
111112

112-
The Superset assets provided by Aspects can be found in the templated `assets.yaml`_ file.
113-
For the most part, these files what Superset exports, but with some crucial differences
113+
The Superset assets provided by Aspects can be found in the templated
114+
`tutoraspects/templates/openedx-assets/assets` directory. For the most part,
115+
these files are what Superset exports, but with some crucial differences
114116
which make these assets usable across all Tutor deployments.
115117

116118
To contribute assets to Aspects:
117119

120+
#. Fork this repository and have a locally running Tutor set up with this plugin
121+
installed.
118122
#. Export the assets you want to contribute as described in `Sharing Charts and Dashboards`
119-
#. Expand the ``.zip`` file.
120-
#. Update any database connection strings to use Tutor configuration template variables
121-
instead of hard-coded strings, e.g. replace ``clickhouse`` with ``{{CLICKHOUSE_HOST}}``.
122-
Passwords can be left as ``{{CLICKHOUSE_PASSWORD}}``, though be aware that if you are adding new
123+
#. Run the command:
124+
`tutor aspects import_superset_zip ~/Downloads/your_file.zip`
125+
#. This command will copy the files from your zip to the assets directory and
126+
attempt to warn you if there are hard coded connection settings where it expects
127+
template variables. These are usually in database and dataset assets, and those are
128+
often assets that already exist. The warnings look like:
129+
130+
`WARN: fact_enrollments.yaml has schema set to reporting instead of a setting.`
131+
#. Check the diff of files and update any database connection strings or table names
132+
to use Tutor configuration template variables instead of hard-coded strings, e.g.
133+
replace ``clickhouse`` with ``{{CLICKHOUSE_HOST}}``. Passwords can be left as
134+
``{{CLICKHOUSE_PASSWORD}}``, though be aware that if you are adding new
123135
databases, you'll need to update ``SUPERSET_DB_PASSWORDS`` in the init scripts.
124136
Here is the default connection string for reference::
125137

126-
``clickhousedb+connect://{{CLICKHOUSE_REPORT_URL}}``
127-
#. Remove any ``metadata.yaml`` files from the export. We generate these as needed during import.
128-
#. Merge your exported files into the directories and files in the `assets.yaml`_.
129-
#. Submit a PR with screenshots of your new chart or dashboards, along with an explanation
130-
of what data question they answer.
138+
``clickhousedb+connect://{{CLICKHOUSE_REPORT_URL}}``
139+
#. You will likely also run into issues where our SQL templates have been expanded into
140+
their actual SQL. If you haven't changed the SQL of these queries (stored in
141+
`tutoraspects/templates/openedx-assets/queries` you can just revert that change back
142+
to their `include` values such as:
143+
`sql: "{% include 'openedx-assets/queries/fact_enrollments_by_day.sql' %}"`
144+
#. The script will also warn about missing `_roles` in dashboards. Superset does not export
145+
these, so you will need to manually add this key with the roles that are necessary to
146+
view the dashboard. See the existing dashboards for how this is done.
147+
#. Run the command `tutor aspects check_superset_assets` to confirm there are no
148+
duplicate assets, which can happen when you rename an asset, and will cause import
149+
to fail. The command will automatically delete the older file if it finds a duplicate.
150+
#. Check that everything imports correctly by running `tutor local do init -l aspects`
151+
and confirming there are no errors.
152+
#. Double check that your database password did not get exported before committing!
153+
#. Commit and submit a PR with screenshots of your new chart or dashboards, along with an
154+
explanation of what data question they answer.
131155

132156

133-
.. _assets.yaml: tutoraspects/templates/aspects/apps/superset/pythonpath/assets.yaml
134-
135157
Virtual datasets in Superset
136158
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
137159

requirements/base.txt

+3-3
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@ kubernetes==27.2.0
3535
# via tutor
3636
markupsafe==2.1.3
3737
# via jinja2
38-
mypy==1.4.1
38+
mypy==1.5.0
3939
# via tutor
4040
mypy-extensions==1.0.0
4141
# via mypy
@@ -63,7 +63,7 @@ pyyaml==6.0.1
6363
# via
6464
# kubernetes
6565
# tutor
66-
regex==2023.6.3
66+
regex==2023.8.8
6767
# via parsimonious
6868
requests==2.31.0
6969
# via
@@ -86,7 +86,7 @@ toolz==0.12.0
8686
# via pyseeyou
8787
transifex-python==3.4.0
8888
# via -r requirements/base.in
89-
tutor==16.0.3
89+
tutor==16.0.5
9090
# via -r requirements/base.in
9191
typing-extensions==4.7.1
9292
# via

requirements/dev.txt

+4-4
Original file line numberDiff line numberDiff line change
@@ -65,7 +65,7 @@ importlib-metadata==6.8.0
6565
# via
6666
# keyring
6767
# twine
68-
importlib-resources==6.0.0
68+
importlib-resources==6.0.1
6969
# via keyring
7070
isort==5.12.0
7171
# via pylint
@@ -99,7 +99,7 @@ mdurl==0.1.2
9999
# via markdown-it-py
100100
more-itertools==10.1.0
101101
# via jaraco-classes
102-
mypy==1.4.1
102+
mypy==1.5.0
103103
# via
104104
# -r requirements/base.txt
105105
# tutor
@@ -171,7 +171,7 @@ pyyaml==6.0.1
171171
# tutor
172172
readme-renderer==40.0
173173
# via twine
174-
regex==2023.6.3
174+
regex==2023.8.8
175175
# via
176176
# -r requirements/base.txt
177177
# parsimonious
@@ -221,7 +221,7 @@ toolz==0.12.0
221221
# pyseeyou
222222
transifex-python==3.4.0
223223
# via -r requirements/base.txt
224-
tutor==16.0.3
224+
tutor==16.0.5
225225
# via -r requirements/base.txt
226226
twine==4.0.2
227227
# via -r requirements/dev.in

requirements/pip-tools.txt

+1-1
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ click==8.1.6
1010
# via pip-tools
1111
packaging==23.1
1212
# via build
13-
pip-tools==7.2.0
13+
pip-tools==7.3.0
1414
# via -r requirements/pip-tools.in
1515
pyproject-hooks==1.0.0
1616
# via build

setup.cfg

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
11
[bumpversion]
2-
current_version = 0.25.0
2+
current_version = 0.26.0
33
commit = False
44
tag = False

tutoraspects/__about__.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -2,4 +2,4 @@
22
Expose some package metadata.
33
"""
44

5-
__version__ = "0.25.0"
5+
__version__ = "0.26.0"

0 commit comments

Comments
 (0)