ValueError: Cannot convert None to Decimal error when passing table_settings

[yoavxyoav]

I am a bit confused about versions.

Ther latest version on Conda an Pip is v0.5.28, but a solution that is (hopefully) solving my problem is mentioning v0.5.8 .

Can anyone please explain this discrepancy and/or how to use version 0.5.8 with Python? Thanks.

[jsvine]

Thanks for your interest in this library @yoavxyoav! If the solution works with v0.5.8 (an earlier version), it should also work with v0.5.28. If it does not, share your code, expected outcome, and result here.

[yoavxyoav]

@jsvine Thank you for the quick reply and for clarifying about the version. I changed the title of this thread to fit my actual problem.

The code I am trying to use is rather simple:

pages = {}
for page_number, page in enumerate(pdf.pages):                      
    page_content = []
    for table in page.extract_tables(table_settings):
        page_content.append(table)
    pages[page_number] = page_content

This gives me the following error: ValueError: Cannot convert None to Decimal.

There is no error when I don't pass table_settings.

which in this case, is actually the default:

table_settings in this case is the default according to the docs:

table_settings = {
    "vertical_strategy": "lines", 
    "horizontal_strategy": "lines",
    "explicit_vertical_lines": [],
    "explicit_horizontal_lines": [],
    "snap_tolerance": 3,
    "join_tolerance": 3,
    "edge_min_length": 3,
    "min_words_vertical": 3,
    "min_words_horizontal": 1,
    "keep_blank_chars": False,
    "text_tolerance": 3,
    "text_x_tolerance": None,
    "text_y_tolerance": None,
    "intersection_tolerance": 3,
    "intersection_x_tolerance": None,
    "intersection_y_tolerance": None,
}

My pdfminer.six version is 20200517

Thanks for your help.

[samkit-jain]

Hi @yoavxyoav The reason you are seeing this behaviour is because of a bug. The issue is happening because of the None value in the fields text_x_tolerance, text_y_tolerance, intersection_x_tolerance and intersection_y_tolerance.

TLDR - If you want to use the default values, don't pass them instead of setting None. For example,

table_settings = {
    "vertical_strategy": "lines", 
    "horizontal_strategy": "lines",
    "explicit_vertical_lines": [],
    "explicit_horizontal_lines": [],
    "snap_tolerance": 3,
    "join_tolerance": 3,
    "edge_min_length": 3,
    "min_words_vertical": 3,
    "min_words_horizontal": 1,
    "keep_blank_chars": False,
    "text_tolerance": 3,
    "intersection_tolerance": 3,
}

Explainer - In the TableFinder class, we clean the table settings provided to resolve null values, but, in the extract_table(...) and extract_tables(...) methods, we are not using the cleaned table settings and using the user-provided None value and since that is being used for comparison against numerical values, you see the error as above. I will be raising a PR shortly to fix this but in the meanwhile, you can use the table settings excluding the null fields.

[samkit-jain]

Handled in #545

[yoavxyoav]

@samkit-jain Thanks for looking into this :)