backport: bitcoin#19801, #20556, #21141, #21235, #21331, #21343, #21424, #21691, #21777

doc/build-osx.md

@@ -1,51 +1,238 @@
-# macOS Build Instructions and Notes
+# macOS Build Guide
+
+**Updated for MacOS [11.2](https://www.apple.com/macos/big-sur/)**
+
+This guide describes how to build dashd, command-line utilities, and GUI on macOS
+
+**Note:** The following is for Intel Macs only!
+
+## Dependencies
+
+The following dependencies are **required**:
+
+Library                                                    | Purpose    | Description
+-----------------------------------------------------------|------------|----------------------
+[automake](https://formulae.brew.sh/formula/automake)      | Build      | Generate makefile
+[libtool](https://formulae.brew.sh/formula/libtool)        | Build      | Shared library support
+[pkg-config](https://formulae.brew.sh/formula/pkg-config)  | Build      | Configure compiler and linker flags
+[boost](https://formulae.brew.sh/formula/boost)            | Utility    | Library for threading, data structures, etc
+[libevent](https://formulae.brew.sh/formula/libevent)      | Networking | OS independent asynchronous networking
+
+The following dependencies are **optional**:
+
+Library                                                         | Purpose          | Description
+--------------------------------------------------------------- |------------------|----------------------
+[berkeley-db@4](https://formulae.brew.sh/formula/berkeley-db@4) | Berkeley DB      | Wallet storage (only needed when wallet enabled)
+[qt@5](https://formulae.brew.sh/formula/qt@5)                   | GUI              | GUI toolkit (only needed when GUI enabled)
+[qrencode](https://formulae.brew.sh/formula/qrencode)           | QR codes in GUI  | Generating QR codes (only needed when GUI enabled)
+[zeromq](https://formulae.brew.sh/formula/zeromq)               | ZMQ notification | Allows generating ZMQ notifications (requires ZMQ version >= 4.0.0)
+[sqlite](https://formulae.brew.sh/formula/sqlite)               | SQLite DB        | Wallet storage (only needed when wallet enabled)
+[miniupnpc](https://formulae.brew.sh/formula/miniupnpc)         | UPnP Support     | Firewall-jumping support (needed for port mapping support)
+[libnatpmp](https://formulae.brew.sh/formula/libnatpmp)         | NAT-PMP Support  | Firewall-jumping support (needed for port mapping support)
+[python3](https://formulae.brew.sh/formula/python@3.9)          | Testing          | Python Interpreter (only needed when running the test suite)
+
+The following dependencies are **optional** packages required for deploying:
+
+Library                                             | Purpose          | Description
+----------------------------------------------------|------------------|----------------------
+[librsvg](https://formulae.brew.sh/formula/librsvg) | Deploy Dependency| Library to render SVG files
+[ds_store](https://pypi.org/project/ds-store/)      | Deploy Dependency| Examine and modify .DS_Store files
+[mac_alias](https://pypi.org/project/mac-alias/)    | Deploy Dependency| Generate/Read binary alias and bookmark records
+
+See [dependencies.md](dependencies.md) for a complete overview.
+
+## Preparation
 
 The commands in this guide should be executed in a Terminal application.
-The built-in one is located in
+macOS comes with a built-in Terminal located in:
+
 ```
 /Applications/Utilities/Terminal.app
 ```
 
-## Preparation
-Install the macOS command line tools:
+### 1. Xcode Command Line Tools
 
-```shell
+The Xcode Command Line Tools are a collection of build tools for macOS.
+These tools must be installed in order to build Dash Core from source.
+
+To install, run the following command from your terminal:
+
+``` bash
 xcode-select --install
 ```
 
-When the popup appears, click `Install`.
+Upon running the command, you should see a popup appear.
+Click on `Install` to continue the installation process.
 
-Then install [Homebrew](https://brew.sh).
+### 2. Homebrew Package Manager
 
-## Dependencies
-```shell
-brew install automake libtool boost gmp miniupnpc pkg-config python qt@5 libevent libnatpmp qrencode
+Homebrew is a package manager for macOS that allows one to install packages from the command line easily.
+While several package managers are available for macOS, this guide will focus on Homebrew as it is the most popular.
+Since the examples in this guide which walk through the installation of a package will use Homebrew, it is recommended that you install it to follow along.
+Otherwise, you can adapt the commands to your package manager of choice.
+
+To install the Homebrew package manager, see: https://brew.sh
+
+Note: If you run into issues while installing Homebrew or pulling packages, refer to [Homebrew's troubleshooting page](https://docs.brew.sh/Troubleshooting).
+
+### 3. Install Required Dependencies
+
+The first step is to download the required dependencies.
+These dependencies represent the packages required to get a barebones installation up and running.
+To install, run the following from your terminal:
+
+``` bash
+brew install automake libtool boost gmp pkg-config libevent
 ```
 
-If you run into issues, check [Homebrew's troubleshooting page](https://docs.brew.sh/Troubleshooting).
-See [dependencies.md](dependencies.md) for a complete overview.
+### 4. Clone Dash repository
 
-The wallet support requires one or both of the dependencies ([*SQLite*](#sqlite) and [*Berkeley DB*](#berkeley-db)) in the sections below.
-To build Dash Core without wallet, see [*Disable-wallet mode*](#disable-wallet-mode).
+`git` should already be installed by default on your system.
+Now that all the required dependencies are installed, let's clone the Dash Core repository to a directory.
+All build scripts and commands will run from this directory.
 
-#### SQLite
+``` bash
+git clone https://github.com/dashpay/dash.git
+```
 
-Usually, macOS installation already has a suitable SQLite installation.
-Also, the Homebrew package could be installed:
+### 5. Install Optional Dependencies
 
-```shell
+#### Wallet Dependencies
+
+It is not necessary to build wallet functionality to run `dashd` or  `dash-qt`.
+To enable legacy wallets, you must install `berkeley-db@4`.
+To enable [descriptor wallets](https://github.com/dashpay/dash/blob/master/doc/descriptors.md), `sqlite` is required.
+Skip `berkeley-db@4` if you intend to *exclusively* use descriptor wallets.
+
+###### Legacy Wallet Support
+
+`berkeley-db@4` is required to enable support for legacy wallets.
+Skip if you don't intend to use legacy wallets.
+
+``` bash
+brew install berkeley-db@4
+```
+
+###### Descriptor Wallet Support
+
+Note: Apple has included a useable `sqlite` package since macOS 10.14.
+You may not need to install this package.
+
+`sqlite` is required to enable support for descriptor wallets.
+Skip if you don't intend to use descriptor wallets.
+
+``` bash
 brew install sqlite
 ```
+---
 
-In that case the Homebrew package will prevail.
+#### GUI Dependencies
 
-If you want to build the disk image with `make deploy` (.dmg / optional), you need:
-[`macdeployqtplus`](../contrib/macdeploy/README.md) dependencies:
-```shell
+###### Qt
+
+Dash Core includes a GUI built with the cross-platform Qt Framework.
+To compile the GUI, we need to install `qt@5`.
+Skip if you don't intend to use the GUI.
+
+``` bash
+brew install qt@5
+```
+
+Note: Building with Qt binaries downloaded from the Qt website is not officially supported.
+See the notes in [#7714](https://github.com/dashpay/dash/issues/7714).
+
+###### qrencode
+
+The GUI can encode addresses in a QR Code. To build in QR support for the GUI, install `qrencode`.
+Skip if not using the GUI or don't want QR code functionality.
+
+``` bash
+brew install qrencode
+```
+---
+
+#### Port Mapping Dependencies
+
+###### miniupnpc
+
+miniupnpc may be used for UPnP port mapping.
+Skip if you do not need this functionality.
+
+``` bash
+brew install miniupnpc
+```
+
+###### libnatpmp
+
+libnatpmp may be used for NAT-PMP port mapping.
+Skip if you do not need this functionality.
+
+``` bash
+brew install libnatpmp
+```
+
+Note: UPnP and NAT-PMP support will be compiled in and disabled by default.
+Check out the [further configuration](#further-configuration) section for more information.
+
+---
+
+#### ZMQ Dependencies
+
+Support for ZMQ notifications requires the following dependency.
+Skip if you do not need ZMQ functionality.
+
+``` bash
+brew install zeromq
+```
+
+ZMQ is automatically compiled in and enabled if the dependency is detected.
+Check out the [further configuration](#further-configuration) section for more information.
+
+For more information on ZMQ, see: [zmq.md](zmq.md)
+
+---
+
+#### Test Suite Dependencies
+
+There is an included test suite that is useful for testing code changes when developing.
+To run the test suite (recommended), you will need to have Python 3 installed:
+
+``` bash
+brew install python
+```
+
+---
+
+#### Deploy Dependencies
+
+You can deploy a `.dmg` containing the Dash Core application using `make deploy`.
+This command depends on a couple of python packages, so it is required that you have `python` installed.
+
+Ensuring that `python` is installed, you can install the deploy dependencies by running the following commands in your terminal:
+
+``` bash
 pip3 install ds_store mac_alias
 ```
 
-#### Berkeley DB
+## Building Dash Core
+
+### 1. Configuration
+
+There are many ways to configure Dash Core, here are a few common examples:
+
+##### Wallet (BDB + SQlite) Support, No GUI:
+
+If `berkeley-db@4` is installed, then legacy wallet support will be built.
+If `berkeley-db@4` is not installed, then this will throw an error.
+If `sqlite` is installed, then descriptor wallet support will also be built.
+Additionally, this explicitly disables the GUI.
+
+``` bash
+./autogen.sh
+./configure --with-gui=no
+```
+
+###### Berkeley DB
 
 It is recommended to use Berkeley DB 4.8. If you have to build it yourself,
 you can use [the installation script included in contrib/](contrib/install_db4.sh)
@@ -55,59 +242,68 @@ like so:
 ./contrib/install_db4.sh .
 ```
 
-from the root of the repository.
+##### Wallet (only SQlite) and GUI Support:
 
-Also, the Homebrew package could be installed:
+This explicitly enables the GUI and disables legacy wallet support.
+If `qt` is not installed, this will throw an error.
+If `sqlite` is installed then descriptor wallet functionality will be built.
+If `sqlite` is not installed, then wallet functionality will be disabled.
 
-```shell
-brew install berkeley-db4
+``` bash
+./autogen.sh
+./configure --without-bdb --with-gui=yes
 ```
 
-## Build Dash Core
+##### No Wallet or GUI
 
-1. Clone the Dash Core source code:
-    ```shell
-    git clone https://github.com/dashpay/dash
-    cd dash
-    ```
+``` bash
+./autogen.sh
+./configure --without-wallet --with-gui=no
+```
 
-2.  Build Dash Core:
+##### Further Configuration
 
-    Configure and build the headless Dash Core binaries as well as the GUI (if Qt is found).
+You may want to dig deeper into the configuration options to achieve your desired behavior.
+Examine the output of the following command for a full list of configuration options:
 
-    You can disable the GUI build by passing `--without-gui` to configure.
-    ```shell
-    ./autogen.sh
-    ./configure
-    make
-    ```
+``` bash
+./configure -help
+```
 
-3.  It is recommended to build and run the unit tests:
-    ```shell
-    make check
-    ```
+### 2. Compile
 
-4.  You can also create a  `.dmg` that contains the `.app` bundle (optional):
-    ```shell
-    make deploy
-    ```
+After configuration, you are ready to compile.
+Run the following in your terminal to compile Dash Core:
 
-## Disable-wallet mode
-When the intention is to run only a P2P node without a wallet, Dash Core may be
-compiled in disable-wallet mode with:
-```shell
-./configure --disable-wallet
+``` bash
+make -jx    # use -jX here for parallelism
+make check  # Run tests if Python 3 is available
 ```
 
-In this case there is no dependency on [*Berkeley DB*](#berkeley-db) and [*SQLite*](#sqlite).
+### 3. Deploy (optional)
 
-Mining is also possible in disable-wallet mode using the `getblocktemplate` RPC call.
+You can also create a  `.dmg` containing the `.app` bundle by running the following command:
 
-## Running
+``` bash
+make deploy
+```
+
+## Running Dash Core
+
+Dash Core should now be available at `./src/dashd`.
+If you compiled support for the GUI, it should be available at `./src/qt/dash-qt`.
 
-Dash Core is now available at `./src/dashd`
+The first time you run `dashd` or `dash-qt`, it will start downloading the blockchain.
+This process could take many hours, or even days on slower than average systems.
+
+By default, blockchain and wallet data files will be stored in:
+
+``` bash
+/Users/${USER}/Library/Application Support/Dash/
+```
 
 Before running, you may create an empty configuration file:
+
 ```shell
 mkdir -p "/Users/${USER}/Library/Application Support/DashCore"
 
@@ -116,23 +312,17 @@ touch "/Users/${USER}/Library/Application Support/DashCore/dash.conf"
 chmod 600 "/Users/${USER}/Library/Application Support/DashCore/dash.conf"
 ```
 
-The first time you run dashd, it will start downloading the blockchain. This process could take many hours, or even days on slower than average systems.
-
 You can monitor the download process by looking at the debug.log file:
+
 ```shell
 tail -f $HOME/Library/Application\ Support/DashCore/debug.log
 ```
 
 ## Other commands:
 
 ```shell
-./src/dashd -daemon      # Starts the dash daemon.
+./src/dashd -daemon      # Starts the dashd daemon.
 ./src/dash-cli --help    # Outputs a list of command-line options.
 ./src/dash-cli help      # Outputs a list of RPC commands when the daemon is running.
+./src/qt/dash-qt -server # Starts the dash-qt server mode, allows dash-cli control
 ```
-
-## Notes
-
-* Tested on OS X 10.12 Sierra through macOS 10.15 Catalina on 64-bit Intel
-processors only.
-* Building with downloaded Qt binaries is not officially supported. See the notes in [#7714](https://github.com/bitcoin/bitcoin/issues/7714).

doc/release-notes-21141.md

@@ -0,0 +1,6 @@
+## Command-line options
+### Changes in existing cmd-line options:
+
+- `-walletnotify=<cmd>` has a new format options "%h" and "%b".
+%b is replaced by the hash of the block including the transaction (set to 'unconfirmed' if the transaction is not included).
+%h is replaced by the block height (-1 if not included).