Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add C/C++ Doxygen docs #22751

Closed
wants to merge 17 commits into from
Closed

Add C/C++ Doxygen docs #22751

wants to merge 17 commits into from

Conversation

tecandrew
Copy link
Contributor

adds C/C++ docs to sphinx-generated docs.comma.ai site.

rel: #22732

@pd0wm
Copy link
Contributor

pd0wm commented Oct 31, 2021

Is it possible to do this without introducing CMake but using scons?

@tecandrew
Copy link
Contributor Author

Is it possible to do this without introducing CMake but using scons?

more than likely, yes. although, I'm not too familiar with setting up dev environments w/ scons 😅

@tecandrew
Copy link
Contributor Author

breathe is having some issues compiling some of the c methods, finding all sorts of weird non-UTF8 characters within OP. just a fyi

#37 212.9 /home/batman/openpilot/docs/build/sphinx/overview.rst:68: WARNING: toctree contains reference to nonexisting document 'tools/sim/README'
#37 212.9 /home/batman/openpilot/docs/build/sphinx/overview.rst:68: WARNING: toctree contains reference to nonexisting document 'tools/ssh/README'
#37 212.9 /home/batman/openpilot/docs/build/sphinx/overview.rst:68: WARNING: toctree contains reference to nonexisting document 'tools/webcam/README'
#37 212.9 
#37 212.9 Exception occurred:
#37 213.0   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/breathe/parser/__init__.py", line 81, in parse
#37 213.0     raise ParserError(e, filename)
#37 213.0 breathe.parser.ParserError: file /home/batman/openpilot/docs/build/doxygen/xml/OMX__QCOMExtns_8h.xml: not well-formed (invalid token): line 5144, column 132
#37 213.0 The full traceback has been saved in /tmp/sphinx-err-on7p8t0o.log, if you want to report the issue to the developers.
 <codeline lineno="148"><highlight class="comment"><sp/><sp/><sp/>*<sp/><sp/>OMX<sp/>IL<sp/>1.1.1<sp/>Figure<sp/>2-11:<sp/>Case<sp/>2<97>Each<sp/>Buffer<sp/>Filled<sp/>with</highlight></codeline>
  /*  2 - Multiple complete frames per buffer (integer number)
   *  OMX IL 1.1.1 Figure 2-11: Case 2�Each Buffer Filled with
   *  Only Complete Frames of Data
   */
  OMX_QCOM_FramePacking_CompleteFrames,

@pd0wm
Copy link
Contributor

pd0wm commented Oct 31, 2021

breathe is having some issues compiling some of the c methods, finding all sorts of weird non-UTF8 characters within OP. just a fyi

#37 212.9 /home/batman/openpilot/docs/build/sphinx/overview.rst:68: WARNING: toctree contains reference to nonexisting document 'tools/sim/README'
#37 212.9 /home/batman/openpilot/docs/build/sphinx/overview.rst:68: WARNING: toctree contains reference to nonexisting document 'tools/ssh/README'
#37 212.9 /home/batman/openpilot/docs/build/sphinx/overview.rst:68: WARNING: toctree contains reference to nonexisting document 'tools/webcam/README'
#37 212.9 
#37 212.9 Exception occurred:
#37 213.0   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/breathe/parser/__init__.py", line 81, in parse
#37 213.0     raise ParserError(e, filename)
#37 213.0 breathe.parser.ParserError: file /home/batman/openpilot/docs/build/doxygen/xml/OMX__QCOMExtns_8h.xml: not well-formed (invalid token): line 5144, column 132
#37 213.0 The full traceback has been saved in /tmp/sphinx-err-on7p8t0o.log, if you want to report the issue to the developers.
 <codeline lineno="148"><highlight class="comment"><sp/><sp/><sp/>*<sp/><sp/>OMX<sp/>IL<sp/>1.1.1<sp/>Figure<sp/>2-11:<sp/>Case<sp/>2<97>Each<sp/>Buffer<sp/>Filled<sp/>with</highlight></codeline>
  /*  2 - Multiple complete frames per buffer (integer number)
   *  OMX IL 1.1.1 Figure 2-11: Case 2�Each Buffer Filled with
   *  Only Complete Frames of Data
   */
  OMX_QCOM_FramePacking_CompleteFrames,

Just exclude the whole third_party folder. We don't want to make changes in that code.

@tecandrew
Copy link
Contributor Author

gotcha. should panda/board/ be ignored as well? another weird character on this line

* list single class (that works) with sphinx-breathe-doxygen
* leave sphinx cmd that refences/builds all c_docs as a comment
@tecandrew
Copy link
Contributor Author

got some good results so far

image

there are some classes breathe just can't interpret nicely. it could be a doxygen issue though.

#37 77.82 /home/batman/openpilot/docs/build/sphinx/overview.rst:83: WARNING: Error when parsing function declaration.
#37 77.82 If the function has no return type:
#37 77.82   Error in declarator or parameters-and-qualifiers
#37 77.82   Invalid C++ declaration: Expected identifier in nested name, got keyword: inline [error at 13]
#37 77.82     inline inline ::cereal::CarParams::LateralPIDTuning::Builder initPid ()
#37 77.82     -------------^
#37 77.82 If the function has a return type:
#37 77.82   Invalid C++ declaration: Expected identifier in nested name, got keyword: inline [error at 13]
#37 77.82     inline inline ::cereal::CarParams::LateralPIDTuning::Builder initPid ()
#37 77.82     -------------^
#37 77.82 
#37 77.82 /home/batman/openpilot/docs/build/sphinx/overview.rst:83: WARNING: Error when parsing function declaration.
#37 77.82 If the function has no return type:
#37 77.82   Error in declarator or parameters-and-qualifiers
#37 77.82   Invalid C++ declaration: Expected identifier in nested name, got keyword: inline [error at 13]
#37 77.82     inline inline ::capnp::Orphan< ::cereal::CarParams::LateralPIDTuning > disownPid ()
#37 77.82     -------------^
#37 77.82 If the function has a return type:
#37 77.82   Invalid C++ declaration: Expected identifier in nested name, got keyword: inline [error at 13]
#37 77.82     inline inline ::capnp::Orphan< ::cereal::CarParams::LateralPIDTuning > disownPid ()
#37 77.82     -------------^
#37 77.82 
#37 77.82 /home/batman/openpilot/docs/build/sphinx/overview.rst:83: WARNING: Error when parsing function declaration.
#37 77.82 If the function has no return type:
#37 77.82   Error in declarator or parameters-and-qualifiers
#37 77.82   Invalid C++ declaration: Expected identifier in nested
#37 77.82 [output clipped, log limit 1MiB reached]

@adeebshihadeh
Copy link
Contributor

got some good results so far

image

there are some classes breathe just can't interpret nicely. it could be a doxygen issue though.

#37 77.82 /home/batman/openpilot/docs/build/sphinx/overview.rst:83: WARNING: Error when parsing function declaration.
#37 77.82 If the function has no return type:
#37 77.82   Error in declarator or parameters-and-qualifiers
#37 77.82   Invalid C++ declaration: Expected identifier in nested name, got keyword: inline [error at 13]
#37 77.82     inline inline ::cereal::CarParams::LateralPIDTuning::Builder initPid ()
#37 77.82     -------------^
#37 77.82 If the function has a return type:
#37 77.82   Invalid C++ declaration: Expected identifier in nested name, got keyword: inline [error at 13]
#37 77.82     inline inline ::cereal::CarParams::LateralPIDTuning::Builder initPid ()
#37 77.82     -------------^
#37 77.82 
#37 77.82 /home/batman/openpilot/docs/build/sphinx/overview.rst:83: WARNING: Error when parsing function declaration.
#37 77.82 If the function has no return type:
#37 77.82   Error in declarator or parameters-and-qualifiers
#37 77.82   Invalid C++ declaration: Expected identifier in nested name, got keyword: inline [error at 13]
#37 77.82     inline inline ::capnp::Orphan< ::cereal::CarParams::LateralPIDTuning > disownPid ()
#37 77.82     -------------^
#37 77.82 If the function has a return type:
#37 77.82   Invalid C++ declaration: Expected identifier in nested name, got keyword: inline [error at 13]
#37 77.82     inline inline ::capnp::Orphan< ::cereal::CarParams::LateralPIDTuning > disownPid ()
#37 77.82     -------------^
#37 77.82 
#37 77.82 /home/batman/openpilot/docs/build/sphinx/overview.rst:83: WARNING: Error when parsing function declaration.
#37 77.82 If the function has no return type:
#37 77.82   Error in declarator or parameters-and-qualifiers
#37 77.82   Invalid C++ declaration: Expected identifier in nested
#37 77.82 [output clipped, log limit 1MiB reached]

This is the generated capnproto code. I think the whole cereal/gen directory can also be skipped.

@tecandrew
Copy link
Contributor Author

added. there are more than a dozen errors when attempting to build and list all c docs within sphinx--I used Thneed class as a test for listing. I ignored those files for the time being as workaround.

also seeing the same error as breathe-doc/breathe#574 when building and referencing all c methods/classes. thinking this is a breathe bug and projects w/ QT dependencies.

#36 28.76 
#36 28.77 Traceback (most recent call last):
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/sphinx/cmd/build.py", line 280, in build_main
#36 28.77     app.build(args.force_all, filenames)
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/sphinx/application.py", line 343, in build
#36 28.77     self.builder.build_update()
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 293, in build_update
#36 28.77     self.build(to_build,
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 307, in build
#36 28.77     updated_docnames = set(self.read())
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 414, in read
#36 28.77     self._read_serial(docnames)
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 435, in _read_serial
#36 28.77     self.read_doc(docname)
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 475, in read_doc
#36 28.77     doctree = read_doc(self.app, self.env, self.env.doc2path(docname))
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/sphinx/io.py", line 189, in read_doc
#36 28.77     pub.publish()
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/docutils/core.py", line 217, in publish
#36 28.77     self.document = self.reader.read(self.source, self.parser,
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/sphinx/io.py", line 109, in read
#36 28.77     self.parse()
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/docutils/readers/__init__.py", line 78, in parse
#36 28.77     self.parser.parse(self.input, document)
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/sphinx/parsers.py", line 101, in parse
#36 28.77     self.statemachine.run(inputlines, document, inliner=self.inliner)
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 170, in run
#36 28.77     results = StateMachineWS.run(self, input_lines, input_offset,
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/docutils/statemachine.py", line 239, in run
#36 28.77     context, next_state, result = self.check_line(
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/docutils/statemachine.py", line 451, in check_line
#36 28.77     return method(match, context, next_state)
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 2769, in underline
#36 28.77     self.section(title, source, style, lineno - 1, messages)
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 327, in section
#36 28.77     self.new_subsection(title, lineno, messages)
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 393, in new_subsection
#36 28.77     newabsoffset = self.nested_parse(
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 281, in nested_parse
#36 28.77     state_machine.run(block, input_offset, memo=self.memo,
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 196, in run
#36 28.77     results = StateMachineWS.run(self, input_lines, input_offset)
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/docutils/statemachine.py", line 239, in run
#36 28.77     context, next_state, result = self.check_line(
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/docutils/statemachine.py", line 451, in check_line
#36 28.77     return method(match, context, next_state)
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 2342, in explicit_markup
#36 28.77     nodelist, blank_finish = self.explicit_construct(match)
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 2354, in explicit_construct
#36 28.77     return method(self, expmatch)
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 2096, in directive
#36 28.77     return self.run_directive(
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 2146, in run_directive
#36 28.77     result = directive_instance.run()
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/breathe/directives/index.py", line 85, in run
#36 28.77     return self.handle_contents(project_info)
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/breathe/directives/index.py", line 55, in handle_contents
#36 28.77     node_list = object_renderer.render(context.node_stack[0], context)
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/breathe/renderer/sphinxrenderer.py", line 2333, in render
#36 28.77     result = method(self, node)  # type: ignore
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/breathe/renderer/sphinxrenderer.py", line 962, in visit_doxygen
#36 28.77     nodelist.extend(self.render(n))
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/breathe/renderer/sphinxrenderer.py", line 2333, in render
#36 28.77     result = method(self, node)  # type: ignore
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/breathe/renderer/sphinxrenderer.py", line 2221, in dispatch_compound
#36 28.77     return self.visit_compound(node)
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/breathe/renderer/sphinxrenderer.py", line 1105, in visit_compound
#36 28.77     return self.visit_class(node)
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/breathe/renderer/sphinxrenderer.py", line 1048, in visit_class
#36 28.77     nodes = self.handle_declaration(nodeDef, declaration, content_callback=content,
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/breathe/renderer/sphinxrenderer.py", line 638, in handle_declaration
#36 28.77     nodes_ = self.run_directive(obj_type, declaration, content_callback, options)
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/breathe/renderer/sphinxrenderer.py", line 597, in run_directive
#36 28.77     nodes = directive.run()
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/sphinx/domains/cpp.py", line 7107, in run
#36 28.77     return super().run()
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/sphinx/directives/__init__.py", line 209, in run
#36 28.77     self.transform_content(contentnode)
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/breathe/renderer/sphinxrenderer.py", line 67, in transform_content
#36 28.77     callback(contentnode)
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/breathe/renderer/sphinxrenderer.py", line 1043, in content
#36 28.77     rendered_data = self.render(file_data, parent_context)
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/breathe/renderer/sphinxrenderer.py", line 2333, in render
#36 28.77     result = method(self, node)  # type: ignore
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/breathe/renderer/sphinxrenderer.py", line 966, in visit_doxygendef
#36 28.77     return self.render(node.compounddef)
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/breathe/renderer/sphinxrenderer.py", line 2333, in render
#36 28.77     result = method(self, node)  # type: ignore
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/breathe/renderer/sphinxrenderer.py", line 1299, in visit_compounddef
#36 28.77     child_nodes = self.render(sectiondef)
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/breathe/renderer/sphinxrenderer.py", line 2333, in render
#36 28.77     result = method(self, node)  # type: ignore
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/breathe/renderer/sphinxrenderer.py", line 1341, in visit_sectiondef
#36 28.77     node_list.extend(self.render_iterable(node.memberdef))
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/breathe/renderer/sphinxrenderer.py", line 2343, in render_iterable
#36 28.77     output.extend(self.render(entry))
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/breathe/renderer/sphinxrenderer.py", line 2333, in render
#36 28.77     result = method(self, node)  # type: ignore
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/breathe/renderer/sphinxrenderer.py", line 2236, in dispatch_memberdef
#36 28.77     return self.visit_variable(node)
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/breathe/renderer/sphinxrenderer.py", line 2041, in visit_variable
#36 28.77     return self.handle_declaration(node, declaration, options=options)
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/breathe/renderer/sphinxrenderer.py", line 638, in handle_declaration
#36 28.77     nodes_ = self.run_directive(obj_type, declaration, content_callback, options)
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/breathe/renderer/sphinxrenderer.py", line 576, in run_directive
#36 28.77     directive = DomainDirectiveFactory.create(self.context.domain, args)
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/breathe/renderer/sphinxrenderer.py", line 324, in create
#36 28.77     cls, name = DomainDirectiveFactory.cpp_classes[args[0]]  # type: ignore
#36 28.77 KeyError: 'property'
#36 28.77 
#36 28.77 Exception occurred:
#36 28.77   File "/root/.pyenv/versions/3.8.10/lib/python3.8/site-packages/breathe/renderer/sphinxrenderer.py", line 324, in create
#36 28.77     cls, name = DomainDirectiveFactory.cpp_classes[args[0]]  # type: ignore
#36 28.77 KeyError: 'property'
#36 28.77 The full traceback has been saved in /tmp/sphinx-err-e4vz80u1.log, if you want to report the issue to the developers.
#36 28.77 Please also report this if it was a user error, so that a better error message can be provided next time.
#36 28.77 A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
#36 30.21 make[3]: *** [sphinx/CMakeFiles/sphinx_build.dir/build.make:62: sphinx/html] Error 2
#36 30.21 make[2]: *** [CMakeFiles/Makefile2:142: sphinx/CMakeFiles/sphinx_build.dir/all] Error 2
#36 30.21 make[1]: *** [CMakeFiles/Makefile2:149: sphinx/CMakeFiles/sphinx_build.dir/rule] Error 2
#36 30.21 make: *** [Makefile:177: sphinx_build] Error 2

@pd0wm
Copy link
Contributor

pd0wm commented Nov 1, 2021

You should probably exclude all moc files generated by Qt

@tecandrew
Copy link
Contributor Author

build weirdly stopped working with some updates. created new PR with c docs and other stuff #23204

@tecandrew tecandrew closed this Dec 13, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants