Database summary statistics export #224
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Lordfirespeed
requested changes
Jul 14, 2025
Comment on lines
1
to
8
| import {PickAttributesToCsvTransform} from "@/routes/applications/data-export/pick-attributes-to-csv-transform"; | ||
| import {UserInfo} from "@/database"; | ||
| import { AgeAugment } from "./user-age-augmenting-transform"; | ||
| import {KeycloakAugments} from "@/lib/keycloak-augmenting-transform"; | ||
| import {ConsentAugments} from "./consent-augmenting-transform"; | ||
| import {AttendanceAugments} from "./attendance-augmenting-transform"; | ||
| import {UserCvAugment} from "@/routes/applications/data-export/user-cv-augmenting-transform"; | ||
| import {FlagAugments} from "@/routes/applications/data-export/user-flag-json-augmentor"; |
There was a problem hiding this comment.
I need to merge the tailwind-v4 branch so we can run the formatter on this, imports are all over the place
| import {UserCvAugment} from "@/routes/applications/data-export/user-cv-augmenting-transform"; | ||
| import {FlagAugments} from "@/routes/applications/data-export/user-flag-json-augmentor"; | ||
|
|
||
| export class AnonymousCsvTransform extends PickAttributesToCsvTransform<UserInfo & AgeAugment & AttendanceAugments & ConsentAugments<"media" | "dsuPrivacy"> & UserCvAugment & FlagAugments<"discipline-of-study" | "dietary-requirement">> { |
There was a problem hiding this comment.
Recommend extracting the type argument here into a type Foo = ... declaration above
Comment on lines
2
to
17
| import {type ChildProcess, exec, type ExecException} from "node:child_process" | ||
| import {createWriteStream} from "node:fs" | ||
| import {mkdir, rm} from "node:fs/promises" | ||
| import {join as pathJoin} from "node:path" | ||
| import {Readable} from "node:stream" | ||
| import {pipeline} from "node:stream/promises" | ||
| import {ServerError} from "@otterhttp/errors" | ||
|
|
||
| import {origin} from "@/config" | ||
| import type {UserInfo} from "@/database" | ||
| import {Group, onlyGroups} from "@/decorators/authorise" | ||
| import {KeycloakAugmentingTransform} from "@/lib/keycloak-augmenting-transform" | ||
| import {getTempDir} from "@/lib/temp-dir" | ||
| import type {Middleware} from "@/types" | ||
|
|
||
| import {hasCode} from "@/lib/type-guards" |
There was a problem hiding this comment.
Your editor has messed with whitespace here - again, formatter would fix it
Comment on lines
22
to
30
| import {AttendanceAugmentingTransform, type AttendanceAugments} from "./attendance-augmenting-transform" | ||
| import {CvExportingWritable} from "./cv-exporting-writable" | ||
| import {FilteringTransform} from "./filtering-transform" | ||
| import {HukCsvTransform} from "./huk-csv-transform" | ||
| import {MlhCsvTransform} from "./mlh-csv-transform" | ||
| import {generateUserCv} from "./user-cv-async-generator" | ||
| import {generateUserInfo} from "./user-info-async-generator" | ||
| import {UserAgeAugmentingTransform} from "@/routes/applications/data-export/user-age-augmenting-transform"; | ||
| import {UserFlagAugmentor} from "@/routes/applications/data-export/user-flag-json-augmentor"; |
| return async (request, response) => { | ||
| const tempDir = await getTempDir(); | ||
| try { | ||
| const fileName: string = "anonymous-data-export.csv" |
There was a problem hiding this comment.
There's no need to annotate this variable's type, since we are assigning it to a string literal - the string type will be inferred
| ageGroupAugment = {ageGroup: "60+"}; | ||
| } | ||
|
|
||
| return {...userInfo, ...ageGroupAugment} |
There was a problem hiding this comment.
Suggested change
| return {...userInfo, ...ageGroupAugment} | |
| return {...userInfo, ageGroup} |
Comment on lines
19
to
20
| {name:"dietary-requirement", label: "dietary_requirements"}, | ||
| {name:"discipline-of-study", label: "discipline_of_study"}, |
There was a problem hiding this comment.
Suggested change
| {name:"dietary-requirement", label: "dietary_requirements"}, | |
| {name:"discipline-of-study", label: "discipline_of_study"}, | |
| {name: "dietary-requirement", label: "dietary_requirements"}, | |
| {name: "discipline-of-study", label: "discipline_of_study"}, |
| {name: "ageGroup", label: "age_group"}, | ||
| {name: "dsuPrivacy", label: "dsu_privacy"}, | ||
| {name: "media", label: "photo_media_consent"}, | ||
| {name: "is_cv_uploaded", label: "has_uploaded_cv"}, |
There was a problem hiding this comment.
Suggested change
| {name: "is_cv_uploaded", label: "has_uploaded_cv"}, | |
| {name: "is_cv_uploaded", label: "did_upload_cv"}, |
| {name: "dsuPrivacy", label: "dsu_privacy"}, | ||
| {name: "media", label: "photo_media_consent"}, | ||
| {name: "is_cv_uploaded", label: "has_uploaded_cv"}, | ||
| {name: "cv_update_time", label: "cv_update_time"}, |
There was a problem hiding this comment.
Suggested change
| {name: "cv_update_time", label: "cv_update_time"}, | |
| {name: "cv_update_time", label: "cv_last_updated_at"}, |
| {name: "media", label: "photo_media_consent"}, | ||
| {name: "is_cv_uploaded", label: "has_uploaded_cv"}, | ||
| {name: "cv_update_time", label: "cv_update_time"}, | ||
| {name: "isCheckedIn", label: "has_attended"} |
There was a problem hiding this comment.
Suggested change
| {name: "isCheckedIn", label: "has_attended"} | |
| {name: "isCheckedIn", label: "attendance"} |
… csv-transformer itself
…strings that can be exported into a CSV
…ew more data points to CSV transform
…nsform to the route handler
eozden-wq
force-pushed
the
feature/anon-csv-export
branch
from
a14a88a to
08213c3
Compare
|
rebased with main branch, ran formatter on each commit |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I have written most of the code which still needs testing. I'm not sure how to configure the handler for the route, and will need help from @Lordfirespeed.
The code also needs to be tested a lot more, and I'm not sure how to set up tests.
Once finished should hopefully close #207