Error with CSV Syncers that previously worked

[DavinRK1]

Last month on the 26th I was testing the searchable metadata tool with a csv syncer and it was working successfully and I have the files in the specific directory. Today I came back to some more testing with other configs and was receiving an error with a new config, as well as the same error with the same inputs that previously worked and got me the metadata files on the 26th. Here is the error:

       INFO     Initializing syncer: <class 'cs_tools_csv_syncer.CSV'>                                      syncer.py:63

╭─────────────────────────────────────────── Traceback (most recent call last) ────────────────────────────────────────────╮
│ C:\Users\XFMJ7YY\AppData\Roaming\cs_tools.cs_tools\Lib\site-packages\cs_tools\cli\commands\main.py:100 in run │
│ │
│ 97 │ _setup_logging() │
│ 98 │ │
│ 99 │ try: │
│ ❱ 100 │ │ return_code = app(standalone_mode=False) │
│ 101 │ │
│ 102 │ except (click.Abort, typer.Abort): │
│ 103 │ │ return_code = 0 │
│ │
│ C:\Users\XFMJ7YY\AppData\Roaming\cs_tools.cs_tools\Lib\site-packages\typer\main.py:326 in call │
│ │
│ C:\Users\XFMJ7YY\AppData\Roaming\cs_tools.cs_tools\Lib\site-packages\typer\main.py:309 in call │
│ │
│ C:\Users\XFMJ7YY\AppData\Roaming\cs_tools.cs_tools\Lib\site-packages\click\core.py:1157 in call │
│ │
│ C:\Users\XFMJ7YY\AppData\Roaming\cs_tools.cs_tools\Lib\site-packages\typer\core.py:723 in main │
│ │
│ ... 4 frames hidden ... │
│ │
│ C:\Users\XFMJ7YY\AppData\Roaming\cs_tools.cs_tools\Lib\site-packages\cs_tools\cli\ux.py:36 in invoke │
│ │
│ 33 │ │ """Hi-jack command execution to fire off dependencies.""" │
│ 34 │ │ with contextlib.ExitStack() as stack: │
│ 35 │ │ │ for dependency in self.dependencies: │
│ ❱ 36 │ │ │ │ stack.enter_context(dependency(ctx)) │
│ 37 │ │ │ │
│ 38 │ │ │ r = ctx.invoke(self.callback, **ctx.params) │
│ 39 │
│ │
│ C:\Program Files\Python312\Lib\contextlib.py:526 in enter_context │
│ │
│ C:\Users\XFMJ7YY\AppData\Roaming\cs_tools.cs_tools\Lib\site-packages\cs_tools\cli\dependencies\syncer.py:64 in │
│ enter │
│ │
│ 61 │ │ │ conf["configuration"]["models"] = self.models │
│ 62 │ │ │
│ 63 │ │ log.info(f"Initializing syncer: {SyncerClass}") │
│ ❱ 64 │ │ self.dict["_syncer"] = SyncerClass(**conf["configuration"]) │
│ 65 │ │
│ 66 │ def exit(self, exc_type, exc_value, exc_traceback): │
│ 67 │ │ if self.is_database_syncer: │
│ │
│ C:\Users\XFMJ7YY\AppData\Roaming\cs_tools.cs_tools\Lib\site-packages\pydantic_settings\main.py:84 in init │
│ │
│ 81 │ │ **values: Any, │
│ 82 │ ) -> None: │
│ 83 │ │ # Uses something other than self the first arg to allow "self" as a settable a │
│ ❱ 84 │ │ super().init( │
│ 86 │ │ │ │ values, │
│ 87 │ │ │ │ _case_sensitive=_case_sensitive, │
│ │
│ C:\Users\XFMJ7YY\AppData\Roaming\cs_tools.cs_tools\Lib\site-packages\pydantic\main.py:171 in init │
│ │
│ 168 │ │ """ │
│ 169 │ │ # __tracebackhide__ tells pytest and some other tools to omit this function fr │
│ 170 │ │ tracebackhide = True │
│ ❱ 171 │ │ self.pydantic_validator.validate_python(data, self_instance=self) │
│ 172 │ │
│ 173 │ # The following line sets a flag that we use to determine when __init__ gets overr │
│ 174 │ init.pydantic_base_init = True # pyright: ignore[reportFunctionMemberAccess │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
ValidationError: 1 validation error for CSV
directory
Field required [type=missing, input_value={}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.6/v/missing

                 ╭───────────────────── 🤷  This is an unhandled error!  🤷 ─────────────────────╮
                 │                                                                               │
                 │ If you encounter this message more than once, please help by letting us know! │
                 │                                                                               │
                 │           GitHub: https://github.com/thoughtspot/cs_tools/issues/new/choose   │
                 │                                                                               │
                 ╰──────────── Run cs_tools logs report to send us your last error. ─────────────╯

PS C:\Users\XFMJ7YY> cs_tools tools searchable metadata --syncer csv://csv_syncer --config Qa
Fetching latest CS Tools release version timed out after 0.33s
[10:40:17] INFO Attempting Basic authentication thoughtspot.py:213
[10:40:18] INFO Initializing syncer: <class 'cs_tools_csv_syncer.CSV'> syncer.py:63
╭─────────────────────────────────────────── Traceback (most recent call last) ────────────────────────────────────────────╮
│ C:\Users\XFMJ7YY\AppData\Roaming\cs_tools.cs_tools\Lib\site-packages\cs_tools\cli\commands\main.py:100 in run │
│ │
│ 97 │ _setup_logging() │
│ 98 │ │
│ 99 │ try: │
│ ❱ 100 │ │ return_code = app(standalone_mode=False) │
│ 101 │ │
│ 102 │ except (click.Abort, typer.Abort): │
│ 103 │ │ return_code = 0 │
│ │
│ C:\Users\XFMJ7YY\AppData\Roaming\cs_tools.cs_tools\Lib\site-packages\typer\main.py:326 in call │
│ │
│ C:\Users\XFMJ7YY\AppData\Roaming\cs_tools.cs_tools\Lib\site-packages\typer\main.py:309 in call │
│ │
│ C:\Users\XFMJ7YY\AppData\Roaming\cs_tools.cs_tools\Lib\site-packages\click\core.py:1157 in call │
│ │
│ C:\Users\XFMJ7YY\AppData\Roaming\cs_tools.cs_tools\Lib\site-packages\typer\core.py:723 in main │
│ │
│ ... 4 frames hidden ... │
│ │
│ C:\Users\XFMJ7YY\AppData\Roaming\cs_tools.cs_tools\Lib\site-packages\cs_tools\cli\ux.py:36 in invoke │
│ │
│ 33 │ │ """Hi-jack command execution to fire off dependencies.""" │
│ 34 │ │ with contextlib.ExitStack() as stack: │
│ 35 │ │ │ for dependency in self.dependencies: │
│ ❱ 36 │ │ │ │ stack.enter_context(dependency(ctx)) │
│ 37 │ │ │ │
│ 38 │ │ │ r = ctx.invoke(self.callback, **ctx.params) │
│ 39 │
│ │
│ C:\Program Files\Python312\Lib\contextlib.py:526 in enter_context │
│ │
│ C:\Users\XFMJ7YY\AppData\Roaming\cs_tools.cs_tools\Lib\site-packages\cs_tools\cli\dependencies\syncer.py:64 in │
│ enter │
│ │
│ 61 │ │ │ conf["configuration"]["models"] = self.models │
│ 62 │ │ │
│ 63 │ │ log.info(f"Initializing syncer: {SyncerClass}") │
│ ❱ 64 │ │ self.dict["_syncer"] = SyncerClass(**conf["configuration"]) │
│ 65 │ │
│ 66 │ def exit(self, exc_type, exc_value, exc_traceback): │
│ 67 │ │ if self.is_database_syncer: │
│ │
│ C:\Users\XFMJ7YY\AppData\Roaming\cs_tools.cs_tools\Lib\site-packages\pydantic_settings\main.py:84 in init │
│ │
│ 81 │ │ **values: Any, │
│ 82 │ ) -> None: │
│ 83 │ │ # Uses something other than self the first arg to allow "self" as a settable a │
│ ❱ 84 │ │ super().init( │
│ 86 │ │ │ │ values, │
│ 87 │ │ │ │ _case_sensitive=_case_sensitive, │
│ │
│ C:\Users\XFMJ7YY\AppData\Roaming\cs_tools.cs_tools\Lib\site-packages\pydantic\main.py:171 in init │
│ │
│ 168 │ │ """ │
│ 169 │ │ # __tracebackhide__ tells pytest and some other tools to omit this function fr │
│ 170 │ │ tracebackhide = True │
│ ❱ 171 │ │ self.pydantic_validator.validate_python(data, self_instance=self) │
│ 172 │ │
│ 173 │ # The following line sets a flag that we use to determine when __init__ gets overr │
│ 174 │ init.pydantic_base_init = True # pyright: ignore[reportFunctionMemberAccess │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
ValidationError: 1 validation error for CSV
directory
Field required [type=missing, input_value={}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.6/v/missing

I will attach the log file to this post as well.

[DavinRK1]

Looks like the text file appended itself to the bottom of the question.

[boonhapus]

@DavinRK1 can you use the declarative form of the syncer? cs_tools tools searchable metadata --syncer "csv://directory=..." --config Qa , replace the ... with the full path of the directory you want to export data to.

For example..

cs_tools tools searchable metadata --syncer "csv://directory=C:\Users\XFMJ7YY\Downloads" --config Qa

[DavinRK1]

This worked, thanks!