v4.8.0

.gitlab-ci.yml

@@ -22,6 +22,8 @@ include:
       file: danger.yaml
 run-danger-mr-linter:
     stage: pre-check
+    tags:
+      - dangerjs
     variables:
       ENABLE_CHECK_UPDATED_CHANGELOG: 'false'
 

.pre-commit-config.yaml

@@ -15,9 +15,9 @@ repos:
     rev: v1.4.1  # the last version running on py3.7
     hooks:
       - id: mypy
-        additional_dependencies: [types-all]
+        additional_dependencies: ['types-PyYAML<=6.0.12.12']
         # ignore wrapper scripts because of name colision with efuse/__init__.py etc.
-        exclude: test/|docs/|espefuse.py|espsecure.py|esptool.py
+        exclude: test/|docs/|espefuse.py|espsecure.py|esptool.py|esp_rfc2217_server.py
   - repo: https://github.com/codespell-project/codespell
     rev: v2.2.5
     hooks:

CHANGELOG.md

@@ -1,3 +1,95 @@
+## v4.8.0 (2024-09-18)
+
+### New Features
+
+- **espefuse**: Supports wafer efuse versions for esp32c61
+- **esptool**: add new command SFDP read
+- **esptool**: Add option to retry connection in a loop
+- **efuse**: Updates efuse table for esp32c5
+- **efuse**: Updates efuse table for esp32p4
+- **esp32c61**: Added stub flasher support
+- **cli**: add autocompletions
+- **esptool**: allow picking UART by VID/PID/Name
+- **esp32c5**: Add USB-serial/JTAG stub support
+- **esp32c5**: Add UART stub support
+- **esptool**: Print key_purpose name for get_security_info cmd
+- **espefuse**: Adds support extend efuse table by user CSV file
+- **espefuse**: Adds efuse dump formats: separated(default) and united(new)
+- **espefuse**: Adds incompatible eFuse settings check for S3
+- **reset**: Apply reconnections to the whole reset sequence, not line transitions
+- **reset**: Automatically reconnect if port disconnects during reset
+- **esp32-p4**: Add ECO1 magic number
+- **espsecure**: Add support for secure boot v2 using ECDSA-P384 signatures
+- **write_flash**: retry flashing if chip disconnects
+- **espefuse**: Allow filtering efuses based on command line arguments
+- **esploader**: Enable context manager for esp instances
+- **espefuse**: Added check for correctness of written data
+- **espefuse**: Improves help for burn_efuse cmd
+- **esp32s3**: clear boot control register on hard reset
+- **esp32-p4**: add spi-connection restriction to ROM class
+- add UF2 IDs for ESP32-C5 and ESP32-C61
+- **espefuse**: Updates efuses for C5 and C61
+- **esp32c61**: add c61 basic flash support (no_stub)
+- **esp32c5**: skipped the stub check for esp32c5 mp
+- **esp32c5**: base support of esp32c5 mp (no stub)
+- Added warning when secure boot enabled
+- **cmds/write_flash**: Recalculated SHA digest for image binary
+- print flash voltage in flash_id command
+- **esptool**: Adds wafer and pkg versions
+- **espefuse**: Update adc_info commands for all chips
+- **espefuse**: Adds new efuses for esp32p4
+- **espefuse**: Allow the espefuse.py to work when coding scheme == 3
+- **err_defs**: Add ROM bootloader flash error definitions
+- Use ruff instead of flake8 and black both in pre-commit and CI
+- **esp32p4**: Enable USB-serial/JTAG in flasher stub
+- **espefuse**: Postpone some efuses to burn them at the very end
+- add advisory port locking
+- **espefuse**: check_error --recover chip even if there are num_errors
+- **espefuse**: Adds new efuses for esp32c6 and esp32h2
+- **esp32c5**: add target esp32c5 beta3
+
+### Bug Fixes
+
+- **esptool**: Fix esp32c61 flash frequency config
+- **esptool**: Fix incorrect chip version for esp32c5
+- **write_flash**: Verify if files will fit against the real flash size when possible
+- **remote_ports**: Disable reset sequence when a socket is used
+- **bitstring**: Restricted bitstring dependency to fix 32-bit compatibility
+- **esp32_d0wdr2_v3**: Print correct chip name
+- pass error message to exception in OTG mode
+- **bin_image**: add check for ELF file segment when saving RAM segments
+- **docs**: Add a note about entering manual bootloader mode
+- **esp32c5**: Fix MAC reading for esptool
+- Erase non-aligned bytes with --no-stub
+- **esp32-c5**: Use a longer reset delay with usb-serial/jtag to stabilize boot-up
+- **espefuse**: Use stub class if stub flasher is running
+- Do not append SHA256 when `--ram-only-header`
+- **elf2image**: add ELF flags to merge condition
+- ram_only_header: pad flash segment to next boundary
+- sort segments if ram_only_header is used
+- **espefuse**: Fix efuse base addr for esp32c5 MP
+- fix type annotation to comply with mypy
+- **espefuse**: Fix burn_key for ECDSA_KEY, it can read pem file
+- **secure_download_mode**: Disable secure boot detection and print more info
+- **esptool**: clear boot control register on ESP32-S3
+- **intelhex**: catch unicode decode errors when converting hex to binary
+- ROM doesn't attach in-package flash chips
+- close file gracefully in espsecure
+- Fixed glitches on RTS line when no_reset option on Windows
+- **merge_bin**: treat files starting with colon as raw files
+- Index image segments from 0 instead of 1
+- **read_flash**: add flash size arg to enable reading past 2MB without stub
+- **read_flash**: flush transmit buffer less often to inrease throughput
+- **esptool**: Proper alignment for SoCs with offset load
+- ignore resetting on unsupported ports
+- **esptool**: Remove the shebang from uf2_writer.py
+
+### Code Refactoring
+
+- Migrated esp_rfc2217_server into standalone subpackage
+- **test/esptool**: Updated tests according to SHA recomputation for binary
+- **style**: Comply with black>=24.0.0
+
 ## v4.7.0 (2023-12-13)
 
 ### New Features

ci/download_flasher_stubs.py

@@ -12,14 +12,15 @@
         "STUB_SET_VERSION": "1",
         "DOWNLOAD_URL": "https://github.com/espressif/esptool-legacy-flasher-stub/releases/download",
         "TAG_URL": "https://github.com/espressif/esptool-legacy-flasher-stub/releases/tag",
-        "VERSION": "v1.2.0",
+        "VERSION": "v1.3.0",
         "FILE_LIST": (
             "esp32",
             "esp32c2",
             "esp32c3",
             "esp32c5",
             "esp32c5beta3",
             "esp32c6",
+            "esp32c61",
             "esp32c6beta",
             "esp32h2",
             "esp32h2beta1",
@@ -50,7 +51,7 @@
     },
 )
 
-DESTINATION_DIR = "esptool/targets/stub_flasher"
+DESTINATION_DIR = os.path.join("esptool", "targets", "stub_flasher")
 
 README_TEMPLATE = """# Licensing
 

docs/en/espefuse/dump-cmd.rst

@@ -12,8 +12,8 @@ Optional arguments:
 
 - ``--format`` - Selects the dump format:
     - ``default`` - Usual console eFuse dump;
-    - ``united`` - All eFuse blocks are stored in one file;
-    - ``separated`` - Each eFuse block is placed in a separate file. The tool will create multiple files based on the given the ``--file_name`` argument. Example: "--file_name /path/blk.bin", blk0.bin, blk1.bin ... blkN.bin. Use the ``burn_block_data`` cmd to write it back to another chip.
+    - ``joint`` - All eFuse blocks are stored in one file;
+    - ``split`` - Each eFuse block is placed in its own file. The tool will create multiple files based on the given the ``--file_name`` argument. Example: "--file_name /path/blk.bin", blk0.bin, blk1.bin ... blkN.bin. Use the ``burn_block_data`` cmd to write it back to another chip.
 - ``--file_name`` - The path to the file in which to save the dump, if not specified, output to the console.
 
 Raw Values Of Efuse Registers
@@ -93,7 +93,7 @@ This command saves dump for each block into a separate file. You need to provide
 
 .. code-block:: none
 
-    > espefuse.py dump --format separated --file_name backup/chip1/blk.bin
+    > espefuse.py dump --format split --file_name backup/chip1/blk.bin
 
     === Run "dump" command ===
     backup/chip1/blk0.bin
@@ -120,7 +120,7 @@ To save all eFuse blocks in one file, use the following command:
 
 .. code-block:: none
 
-    > espefuse.py dump --format united --file_name backup/chip1/efuses.bin
+    > espefuse.py dump --format joint --file_name backup/chip1/efuses.bin
 
     === Run "dump" command ===
     backup/chip1/efuses.bin

docs/en/esptool/advanced-commands.rst

@@ -120,6 +120,18 @@ A second option ``--non-volatile`` can be used in order to send a ``WREN`` (06h)
 
     Setting status bits (particularly non-volatile ones) can have permanent side effects for some flash chips, so check carefully before using this command to set any bits!
 
+.. _read-flash-sfdp:
+
+Read Serial Flash Discoverable Parameters (SFDP)
+------------------------------------------------
+
+The Serial Flash Discoverable Parameters (SFDP) store essential vendor-specific configuration data of the flash memory chip. These parameters help identify and interact with different flash devices. Usage:
+
+::
+    esptool.py read_flash_sfdp 16 4
+
+This will read 4 bytes from SFDP address 16.
+
 .. only:: esp8266
 
     .. _chip-id:

docs/en/esptool/advanced-options.rst

@@ -36,6 +36,27 @@ The ``--after`` argument allows you to specify whether the chip should be reset
     * ``--after no_reset`` leaves the chip in the serial bootloader, no reset is performed.
     * ``--after no_reset_stub`` leaves the chip in the stub bootloader, no reset is performed.
 
+
+Connect Loop
+------------
+
+Esptool supports connection loops, where the user can specify how many times to try to open a port. The delay between retries is 0.1 seconds. This can be useful for example when the chip is in deep sleep or esptool was started before the chip was connected to the PC. A connection loop can be created by setting the ``ESPTOOL_OPEN_PORT_ATTEMPTS`` environment variable.
+This feature can also be enabled by using the ``open_port_attempts`` configuration option, for more details regarding config options see :ref:`Configuration file <config>` section.
+There are 3 possible values for this option:
+
+.. list::
+
+    * ``0`` will keep trying to connect to the chip indefinitely
+    * ``1`` will try to connect to the chip only once (default)
+    * ``N`` will try to connect to the chip N times
+
+
+.. note::
+
+    This option is only available if both the ``--port`` and ``--chip`` arguments are set.
+
+
+
 .. _disable_stub:
 
 Disabling the Stub Loader

docs/en/esptool/basic-commands.rst

@@ -338,6 +338,7 @@ The following commands are less commonly used, or only of interest to advanced u
     *  :ref:`read-mem-write-mem`
     *  :ref:`read-flash-status`
     *  :ref:`write-flash-status`
+    *  :ref:`read-flash-sfdp`
     :esp8266: *  :ref:`chip-id`
     :esp8266: *  :ref:`make-image`
     :esp8266: *  :ref:`run`

docs/en/esptool/configuration-file.rst

@@ -78,7 +78,7 @@ Sample configuration file:
 Options
 -------
 
-Complete list configurable options:
+Complete list of configurable options:
 
 +------------------------------+-----------------------------------------------------------+----------+
 | Option                       | Description                                               | Default  |
@@ -107,6 +107,8 @@ Complete list configurable options:
 +------------------------------+-----------------------------------------------------------+----------+
 | reset_delay                  | Time to wait before the boot pin is released after reset  | 0.05 s   |
 +------------------------------+-----------------------------------------------------------+----------+
+| open_port_attempts           | Number of attempts to open the port (0 - infinite)        | 1        |
++------------------------------+-----------------------------------------------------------+----------+
 | custom_reset_sequence        | Custom reset sequence for resetting into the bootloader   |          |
 +------------------------------+-----------------------------------------------------------+----------+
 

docs/en/installation.rst

@@ -68,3 +68,41 @@ If updating directly is unavoidable, make sure you update to a compatible versio
 ::
 
    $ pip install esptool==3.3.2
+
+Shell completions
+-----------------
+
+To activate autocompletion, you can manually add commands provided below to your shell's config file
+or run them in your current terminal session for one-time activation.
+You will likely have to restart or re-login for the autocompletion to start working.
+
+bash:
+::
+
+   eval "$(register-python-argcomplete esptool.py espsecure.py espefuse.py)"
+
+zsh:
+
+To activate completions in zsh, first make sure `compinit` is marked for
+autoload and run autoload:
+
+.. code-block:: bash
+
+   autoload -U compinit
+   compinit
+
+Afterwards you can enable completions for esptool.py, espsecure.py and espefuse.py:
+
+::
+
+   eval "$(register-python-argcomplete esptool.py espsecure.py espefuse.py)"
+
+fish:
+
+Not required to be in the config file, only run once
+
+::
+
+   register-python-argcomplete --shell fish esptool.py espsecure.py espefuse.py >~/.config/fish/completions/esptool.py.fish
+
+Other shells nor OS Windows are not supported.

docs/en/troubleshooting.rst

@@ -106,14 +106,17 @@ Early Stage Crash
 
 .. only:: not esp8266 and not esp32 and not esp32c2
 
-   Issues When Using USB-Serial/JTAG or USB-OTG
-   --------------------------------------------
+   Issues and Debugging in USB-Serial/JTAG or USB-OTG modes
+   --------------------------------------------------------
 
-   When working with ESP chips that implement a `USB-Serial/JTAG Controller <https://docs.espressif.com/projects/esp-idf/en/latest/esp32c3/api-guides/usb-serial-jtag-console.html>`_ or a `USB-OTG console <https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/api-guides/usb-otg-console.html>`_, it's essential to be aware of potential issues related to the loaded application interfering with or reprogramming the GPIO pins used for USB communication.
+   When working with ESP chips that implement a `USB-Serial/JTAG <https://docs.espressif.com/projects/esp-idf/en/latest/esp32c3/api-guides/usb-serial-jtag-console.html>`_ or a `USB-OTG <https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/api-guides/usb-otg-console.html>`_ console (you are not using a classic USB-to-Serial adapter), it's essential to be aware of potential issues related to the loaded application interfering with or reprogramming the GPIO pins used for USB communication.
 
    If the application accidentally reconfigures the USB peripheral pins or disables the USB peripheral, the device disappears from the system. You can also encounter unstable flashing or errors like ``OSError: [Errno 71] Protocol error``.
 
-   If that happens, try :ref:`manually entering the download mode <manual-bootloader>` and then using the :ref:`erase_flash <erase_flash>` command to wipe the flash memory. Then, make sure to fix the issue in the application before flashing again.
+   If that happens, try to :ref:`manually enter the download mode <manual-bootloader>` and then use the :ref:`erase_flash <erase_flash>` command to wipe the flash memory. Then, make sure to fix the issue in the application before flashing again.
+
+   On boards with two USB ports (usually marked as USB and UART), you can use the USB port for flashing while listening on the UART port for debugging purposes. This setup is useful for retrieving core dumps or the reset reason in the event of a crash. To implement this, connect the UART port to another instance of any of the `serial terminal programs`_, while repeating the failing action over the USB port. You'll be able to monitor the crash log without interference from the USB port used for communication or it disappearing due to a firmware crash.
+   If your devkit doesn't have a dedicated USB port connected to an on-board USB-to-UART bridge, you can use a separate adapter to connect to the UART pins on the board.
 
 Serial Terminal Programs
 ------------------------
@@ -197,15 +200,7 @@ A serial exception error occurred
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 ``esptool.py`` uses the `pySerial <https://pyserial.readthedocs.io/en/latest/>`_ Python module for accessing the serial port.
-If pySerial cannot operate normally, it raises an error and terminates. Some of the most common pySerial error causes are:
-
-.. list::
-
-   * You don't have permission to access the port.
-   * The port is being already used by other software.
-   * The port doesn't exist.
-   * The device gets unexpectedly disconnected.
-   * The necessary serial port drivers are not installed or are faulty.
+If pySerial cannot operate normally, it raises an error and terminates.
 
 An example of a pySerial error:
 
@@ -214,3 +209,15 @@ An example of a pySerial error:
    A serial exception error occurred: read failed: [Errno 6] Device not configured
 
 Errors originating from pySerial are, therefore, not a problem with ``esptool.py``, but are usually caused by a problem with hardware or drivers.
+
+Some of the most common pySerial error causes are:
+
+.. list::
+
+   * The port is being already used by other software.
+   * The port doesn't exist.
+   * The device gets unexpectedly disconnected.
+   * The necessary serial port drivers are not installed or are faulty.
+   * You don't have permission to access the port.
+
+On Linux, read and write access the serial port over USB is necessary. You can add your user to the ``dialout`` or ``uucp`` group to grant access to the serial port. See `Adding user to dialout or uucp on Linux <https://docs.espressif.com/projects/esp-idf/en/stable/get-started/establish-serial-connection.html#adding-user-to-dialout-or-uucp-on-linux>`_.