Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

2213 custom connector front end #2997

Merged
merged 24 commits into from
Apr 10, 2023
Merged
Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
4c6d526
Refactoring connector registry to prep for connector template uploads
galvana Mar 20, 2023
117c92b
Removing unused import
galvana Mar 20, 2023
69ce2f1
Restoring docstrings
galvana Mar 20, 2023
d7663b4
Adding custom connector template model and custom connector template …
galvana Mar 23, 2023
3774360
Merging in main
galvana Mar 23, 2023
5609182
Fixing mypy and pylint issues
galvana Mar 23, 2023
8d954b8
Fixing down_revision for migration script
galvana Mar 23, 2023
ab7b35d
Updating db_dataset to include new table
galvana Mar 23, 2023
e239b85
Updating sample connector name to prevent confusion
galvana Mar 27, 2023
8b3fe92
Merging in main
galvana Mar 27, 2023
1ec11d5
Adding endpoint for connector template registration
galvana Mar 28, 2023
fcabb3c
Fixing misc issues
galvana Mar 29, 2023
c01f24f
Merge branch 'main' into 2213-custom-connector-template-endpoint
galvana Mar 29, 2023
8d109f2
Fixing static issues
galvana Mar 29, 2023
18cb408
Adding replaceable flag to connector templates plus additional tests
galvana Mar 30, 2023
aebbb38
Merge branch 'main' into 2213-custom-connector-template-endpoint
galvana Mar 30, 2023
0877945
First draft of the custom connector front-end
galvana Apr 5, 2023
9496277
Merging in main
galvana Apr 5, 2023
1619d95
Fixing eslint issues and adding Restrict around upload connector button
galvana Apr 5, 2023
f9e2638
Reverting accidental changes
galvana Apr 5, 2023
9ebab46
Fixing imports and removing unnecessary code
galvana Apr 7, 2023
ffd6cf7
Updating changelog
galvana Apr 7, 2023
09e213a
Merge branch 'main' into 2213-custom-connector-front-end
galvana Apr 7, 2023
5211f42
Merging in main
galvana Apr 10, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions clients/admin-ui/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@
"react-dnd": "^15.0.1",
"react-dnd-html5-backend": "^16.0.1",
"react-dom": "^17.0.2",
"react-dropzone": "^14.2.3",
"react-redux": "^8.0.5",
"react-table": "^7.8.0",
"redux-persist": "^6.0.0",
Expand Down
1 change: 1 addition & 0 deletions clients/admin-ui/src/constants.ts
Original file line number Diff line number Diff line change
Expand Up @@ -136,3 +136,4 @@ export const INDEX_ROUTE = "/";
export const LOGIN_ROUTE = "/login";
export const CONNECTION_ROUTE = "/connection";
export const CONNECTION_TYPE_ROUTE = "/connection_type";
export const CONNECTOR_TEMPLATE = "/connector_template";
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
import {
Box,
Button,
ButtonGroup,
Modal,
ModalBody,
ModalContent,
ModalFooter,
ModalHeader,
ModalOverlay,
Text,
useToast,
} from "@fidesui/react";
import { FetchBaseQueryError } from "@reduxjs/toolkit/dist/query/fetchBaseQuery";
import React, { useState } from "react";
import { useDropzone } from "react-dropzone";

import { getErrorMessage } from "../common/helpers";
galvana marked this conversation as resolved.
Show resolved Hide resolved
import { errorToastParams, successToastParams } from "../common/toast";
import { useRegisterConnectorTemplateMutation } from "./connector-template.slice";

type RequestModalProps = {
isOpen: boolean;
onClose: () => void;
testId?: String;
};

const ConnectorTemplateUploadModal: React.FC<RequestModalProps> = ({
isOpen,
onClose,
testId = "connector-template-modal",
}) => {
const [uploadedFile, setUploadedFile] = useState<File | null>(null);
const toast = useToast();
const { getRootProps, getInputProps, isDragActive } = useDropzone({
onDrop: (acceptedFiles: File[]) => {
const file = acceptedFiles[0];
const fileExtension = file.name.split(".").pop()?.toLowerCase();

if (fileExtension !== "zip") {
toast(errorToastParams("Only zip files are allowed."));
return;
}

setUploadedFile(acceptedFiles[0]);
},
});

const [registerConnectorTemplate, { isLoading }] =
useRegisterConnectorTemplateMutation();

const handleSubmit = async () => {
if (uploadedFile) {
try {
await registerConnectorTemplate(uploadedFile).unwrap();
toast(successToastParams("Connector template uploaded successfully."));
onClose();
} catch (error) {
toast(errorToastParams(getErrorMessage(error as FetchBaseQueryError)));
} finally {
setUploadedFile(null);
}
}
};

const renderFileText = () => {
if (uploadedFile) {
return <Text>{uploadedFile.name}</Text>;
}
if (isDragActive) {
return <Text>Drop the file here...</Text>;
}
return <Text>Click or drag and drop your file here.</Text>;
};

return (
<Modal isOpen={isOpen} onClose={onClose} size="2xl">
<ModalOverlay />
<ModalContent textAlign="left" p={2} data-testid={testId}>
<ModalHeader>Upload connector template</ModalHeader>
<ModalBody>
<Text fontSize="sm" mb={4}>
Drag and drop your connector template zip file here, or click to
browse your files.
</Text>
<Box
{...getRootProps()}
bg={isDragActive ? "gray.100" : "gray.50"}
border="2px dashed"
borderColor={isDragActive ? "gray.300" : "gray.200"}
borderRadius="md"
cursor="pointer"
minHeight="150px"
display="flex"
alignItems="center"
justifyContent="center"
textAlign="center"
>
<input {...getInputProps()} />
{renderFileText()}
</Box>
<Text fontSize="sm" mt={4}>
A connector template zip file must include a SaaS config and
dataset, but may also contain an icon (.svg) and custom functions
(.py) as optional files.
</Text>
</ModalBody>
<ModalFooter>
<ButtonGroup
size="sm"
spacing="2"
width="100%"
display="flex"
justifyContent="right"
>
<Button
variant="outline"
onClick={onClose}
data-testid="cancel-btn"
isDisabled={isLoading}
>
Cancel
</Button>
<Button
colorScheme="primary"
type="submit"
isDisabled={!uploadedFile || isLoading}
onClick={handleSubmit}
data-testid="submit-btn"
>
Submit
</Button>
</ButtonGroup>
</ModalFooter>
</ModalContent>
</Modal>
);
};

export default ConnectorTemplateUploadModal;
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
import { createSlice, PayloadAction } from "@reduxjs/toolkit";

import { CONNECTOR_TEMPLATE } from "~/constants";
import { baseApi } from "~/features/common/api.slice";

import { ConnectorTemplateState } from "./types";

const initialState: ConnectorTemplateState = {
loading: false,
error: null,
};

export const connectorTemplateSlice = createSlice({
name: "connectorTemplate",
initialState,
reducers: {
setLoading: (draftState, action: PayloadAction<boolean>) => {
draftState.loading = action.payload;
},
setError: (draftState, action: PayloadAction<string | null>) => {
draftState.error = action.payload;
},
galvana marked this conversation as resolved.
Show resolved Hide resolved
},
});

export const { setLoading, setError } = connectorTemplateSlice.actions;
export const { reducer } = connectorTemplateSlice;

export const connectorTemplateApi = baseApi.injectEndpoints({
endpoints: (build) => ({
registerConnectorTemplate: build.mutation<void, File>({
query: (file) => {
const formData = new FormData();
formData.append("file", file);

return {
url: `${CONNECTOR_TEMPLATE}/register`,
method: "POST",
body: formData,
};
},
}),
}),
});

export const { useRegisterConnectorTemplateMutation } = connectorTemplateApi;
4 changes: 4 additions & 0 deletions clients/admin-ui/src/features/connector-templates/types.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
export interface ConnectorTemplateState {
loading: boolean;
error: string | null;
}
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import {
Box,
Button,
Center,
Flex,
Input,
Expand All @@ -15,11 +16,20 @@ import {
setSearch,
useGetAllConnectionTypesQuery,
} from "connection-type/connection-type.slice";
import React, { useCallback, useEffect, useMemo, useRef } from "react";
import React, {
useCallback,
useEffect,
useMemo,
useRef,
useState,
} from "react";
import { useDispatch } from "react-redux";

import { useAppSelector } from "~/app/hooks";
import Restrict from "~/features/common/Restrict";
import { ScopeRegistryEnum } from "~/types/api";

import ConnectorTemplateUploadModal from "../../connector-templates/ConnectorTemplateUploadModal";
galvana marked this conversation as resolved.
Show resolved Hide resolved
import Breadcrumb from "./Breadcrumb";
import ConnectionTypeFilter from "./ConnectionTypeFilter";
import ConnectionTypeList from "./ConnectionTypeList";
Expand All @@ -32,6 +42,7 @@ const ChooseConnection: React.FC = () => {
const filters = useAppSelector(selectConnectionTypeFilters);
const { data, isFetching, isLoading, isSuccess } =
useGetAllConnectionTypesQuery(filters);
const [isModalOpen, setIsModalOpen] = useState(false);

const handleSearchChange = useCallback(
(event: React.ChangeEvent<HTMLInputElement>) => {
Expand All @@ -56,6 +67,10 @@ const ChooseConnection: React.FC = () => {
[data]
);

const handleUploadButtonClick = () => {
setIsModalOpen(true);
};

useEffect(() => {
mounted.current = true;
return () => {
Expand Down Expand Up @@ -95,7 +110,23 @@ const ChooseConnection: React.FC = () => {
type="search"
/>
</InputGroup>
<Restrict scopes={[ScopeRegistryEnum.CONNECTOR_TEMPLATE_REGISTER]}>
galvana marked this conversation as resolved.
Show resolved Hide resolved
<Button
colorScheme="primary"
type="submit"
width="20%"
galvana marked this conversation as resolved.
Show resolved Hide resolved
data-testid="upload-btn"
size="sm"
onClick={handleUploadButtonClick}
>
Upload connector
</Button>
</Restrict>
</Flex>
<ConnectorTemplateUploadModal
isOpen={isModalOpen}
onClose={() => setIsModalOpen(false)}
/>
{(isFetching || isLoading) && (
<Center>
<Spinner />
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,19 +3,19 @@ import { useAPIHelper } from "common/hooks";
import { useAlert } from "common/hooks/useAlert";
import {
selectConnectionTypeState,
setConnection,
setConnection
} from "connection-type/connection-type.slice";
import { ConnectionTypeSecretSchemaReponse } from "connection-type/types";
import { SaasType } from "datastore-connections/constants";
import {
useCreateSassConnectionConfigMutation,
usePatchDatastoreConnectionMutation,
useUpdateDatastoreConnectionSecretsMutation,
useUpdateDatastoreConnectionSecretsMutation
} from "datastore-connections/datastore-connection.slice";
import {
CreateSassConnectionConfigRequest,
CreateSaasConnectionConfigRequest,
DatastoreConnectionRequest,
DatastoreConnectionSecretsRequest,
DatastoreConnectionSecretsRequest
} from "datastore-connections/types";
import React, { useState } from "react";
import { useDispatch } from "react-redux";
Expand Down Expand Up @@ -105,7 +105,7 @@ export const ConnectorParameters: React.FC<ConnectorParametersProps> = ({
}
} else {
// Create new Sass connector
const params: CreateSassConnectionConfigRequest = {
const params: CreateSaasConnectionConfigRequest = {
description: values.description,
name: values.name,
instance_key: formatKey(values.instance_key as string),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@ import { DisabledStatus, TestingStatus } from "./constants";
import {
CreateAccessManualWebhookRequest,
CreateAccessManualWebhookResponse,
CreateSassConnectionConfigRequest,
CreateSassConnectionConfigResponse,
CreateSaasConnectionConfigRequest,
CreateSaasConnectionConfigResponse,
DatastoreConnection,
DatastoreConnectionParams,
DatastoreConnectionRequest,
Expand Down Expand Up @@ -164,8 +164,8 @@ export const datastoreConnectionApi = baseApi.injectEndpoints({
invalidatesTags: () => ["DatastoreConnection"],
}),
createSassConnectionConfig: build.mutation<
CreateSassConnectionConfigResponse,
CreateSassConnectionConfigRequest
CreateSaasConnectionConfigResponse,
CreateSaasConnectionConfigRequest
>({
query: (params) => ({
url: `${CONNECTION_ROUTE}/instantiate/${params.saas_connector_type}`,
Expand Down
4 changes: 2 additions & 2 deletions clients/admin-ui/src/features/datastore-connections/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -208,7 +208,7 @@ export type SaasConfig = {
type: SaasType;
};

export type CreateSassConnectionConfigRequest = {
export type CreateSaasConnectionConfigRequest = {
name: string;
description: string;
instance_key: string;
Expand All @@ -218,7 +218,7 @@ export type CreateSassConnectionConfigRequest = {
};
};

export type CreateSassConnectionConfigResponse = {
export type CreateSaasConnectionConfigResponse = {
connection: DatastoreConnection;
dataset: {
fides_key: string;
Expand Down
1 change: 1 addition & 0 deletions clients/admin-ui/src/types/api/models/ScopeRegistryEnum.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ export enum ScopeRegistryEnum {
CONNECTION_READ = "connection:read",
CONNECTION_AUTHORIZE = "connection:authorize",
CONNECTION_TYPE_READ = "connection_type:read",
CONNECTOR_TEMPLATE_REGISTER = "connector_template:register",
CONSENT_READ = "consent:read",
CTL_DATASET_CREATE = "ctl_dataset:create",
CTL_DATASET_READ = "ctl_dataset:read",
Expand Down