remove unnecessary database queries #1324
Conversation
pyiron_base/project/generic.py
Outdated
| if job_id not in self.get_job_ids(recursive=recursive): | ||
| continue |
There was a problem hiding this comment.
If I understand this pull request correctly, then the primary change is removing this database query, correct? Maybe we should split this pull request in two parts. One containing just the removal of this part and the other the refactoring. Having both mixed makes it a bit hard to read.
| state.logger.debug( | ||
| "hdf file does not exist. Removal from database will be attempted." | ||
| ) | ||
| self.db.delete_item(job.id) |
There was a problem hiding this comment.
@jan-janssen You missed this line. It might not be called often but it is another redundant database query.
There was a problem hiding this comment.
But this is dangerous. For example when you create a job object and remove the HDF5 file, then the loading of the job object with inspect() fails and returns an IOError still in the except case the job object is not available, so we cannot use job.id.
There was a problem hiding this comment.
Ok, I realised that the inspect() function also works when the file is corrupted #1327
Improves performance of
remove_jobs.