From 8d2c30ca7328b4d70d1ec5a93ef4eaeac666dea2 Mon Sep 17 00:00:00 2001 From: Wyatt Marchand Date: Wed, 26 Aug 2026 09:28:47 -0700 Subject: [PATCH] esp32s3-eth programming setup guide --- ...ject - Empty Directory to Ready-to-Code.md | 295 ++++++++++++++++++ ...ment for Waveshare ESP32-S3-ETH (W5500).md | 196 ------------ 2 files changed, 295 insertions(+), 196 deletions(-) create mode 100644 Projects/ESP32-S3-ETH/ESP32-S3-ETH Rust Project - Empty Directory to Ready-to-Code.md delete mode 100644 Projects/ESP32-S3-ETH/Rust Dev Environment for Waveshare ESP32-S3-ETH (W5500).md diff --git a/Projects/ESP32-S3-ETH/ESP32-S3-ETH Rust Project - Empty Directory to Ready-to-Code.md b/Projects/ESP32-S3-ETH/ESP32-S3-ETH Rust Project - Empty Directory to Ready-to-Code.md new file mode 100644 index 0000000..3c2b222 --- /dev/null +++ b/Projects/ESP32-S3-ETH/ESP32-S3-ETH Rust Project - Empty Directory to Ready-to-Code.md @@ -0,0 +1,295 @@ +End-to-end setup for a Waveshare ESP32-S3-ETH board (W5500 ethernet over SPI), using the `std` Rust toolchain (`esp-idf-svc`). By the end you'll have a project that connects to ethernet, gets a DHCP lease, and is structured to build real functionality on top of. + + This walkthough was AI generated + +--- + +## 0. Hardware reference + +Confirm these against your board's silkscreen before wiring anything extra — Waveshare's pin assignments have varied slightly across production batches. + +|Signal|GPIO| +|---|---| +|SCLK|13| +|MOSI (SDO)|11| +|MISO (SDI)|12| +|CS|14| +|INT|10| +|RST|9| +|SPI host|SPI2| +|SPI clock|20–25 MHz| + +The W5500 has no burned-in MAC address — you supply one in software (any locally-administered address, e.g. `02:00:00:12:34:56`, works). + +--- + +## 1. Prerequisites + +- USB-C cable, board connected to your dev machine. +- ~5GB free disk space (toolchain + ESP-IDF + submodules). +- Linux/macOS/WSL. (Windows native works too but paths below assume a POSIX shell.) +- `git`, a C compiler, `python3`, and standard build tools already on the system (`build-essential` on Debian/Ubuntu, or equivalent). + +--- + +## 2. Install the Rust + Espressif toolchain + +```bash +# Rust itself, if not already installed +curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh + +# espup: installs the Xtensa-patched Rust toolchain ESP32-S3 needs +cargo install espup +cargo install ldproxy +espup install + +# Load the Xtensa toolchain into your shell (needed every new shell) +. $HOME/export-esp.sh +``` + +Add the `. $HOME/export-esp.sh` line to your shell rc file (`.bashrc`/`.zshrc`/`config.fish`) so you don't have to re-run it manually every session. + +Verify the Xtensa target is present: + +```bash +rustc --print target-list | grep xtensa +``` + +You should see `xtensa-esp32s3-none-elf` and similar in the output. + +--- + +## 3. Scaffold the project + +Use `esp-idf-template`, **not** `esp-generate` — the latter scaffolds `no_std`/`esp-hal` projects, which have no W5500/ethernet driver and no standard HTTP server story. + +```bash +cargo install cargo-generate +cargo generate esp-rs/esp-idf-template cargo +``` + +You'll be prompted interactively. Answers for this board/use case: + +|Prompt|Answer|Why| +|---|---|---| +|Project name|your choice|| +|MCU|`esp32s3`|matches this board| +|Configure advanced options?|Yes|need to reach the STD toggle below| +|STD support|**Yes**|pulls in `esp-idf-svc`; without this you're back on `no_std`| +|ESP-IDF version|`v5.3.4`|most stable match to current esp-idf-svc/hal crates; avoid `master`| +|Use git (not crates.io) esp-idf-* crates|**false**|crates.io releases are stable and documented; git HEAD drifts underneath you| +|Installation location of managed ESP-IDF|**global**|shared across projects, avoids re-downloading ~1–2GB per project| +|Configure Dev Containers?|false|flashing real hardware over USB is simpler done natively| +|Configure Wokwi simulation?|false|Wokwi doesn't model the W5500/ethernet peripheral| +|Add CI files?|false|add later if/when you push to GitHub| + +This produces: + +``` +your-project/ +├── .cargo/config.toml +├── Cargo.toml +├── build.rs +├── rust-toolchain.toml +├── sdkconfig.defaults +└── src/main.rs +``` + +--- + +## 4. Confirm the generated build target + +Open `.cargo/config.toml` and confirm: + +```toml +[build] +target = "xtensa-esp32s3-espidf" +``` + +If this is missing or wrong, `cargo build` will silently try to compile for your host machine instead of the chip and fail with `Unsupported target 'x86_64-unknown-linux-gnu'` (or similar). The template sets this correctly by default — just verify it. + +--- + +## 5. Configure sdkconfig.defaults + +Edit `sdkconfig.defaults` (created empty or near-empty by the template) to include: + +``` +CONFIG_ETH_SPI_ETHERNET_W5500=y +CONFIG_ESP_MAIN_TASK_STACK_SIZE=8000 +CONFIG_HTTPD_MAX_REQ_HDR_LEN=1024 +``` + +The first line is required — without it, ESP-IDF doesn't build the W5500 driver component in at all, and your ethernet code will fail to link or fail at runtime. The other two are reasonable general-purpose values (headroom for the main task stack, generous HTTP header limit if you add a server later). + +--- + +## 6. Configure Cargo.toml + +Replace the `[dependencies]` section with at minimum: + +```toml +[dependencies] +esp-idf-svc = { version = "0.51", features = ["std"] } +anyhow = "1" +log = "0.4" +``` + +Add more crates as your project needs them — see the companion note on porting non-embedded dependencies if you're bringing code over from another platform. + +--- + +## 7. First build (this installs ESP-IDF itself) + +```bash +cargo build --release +``` + +The **first** build does a lot of one-time work: downloads ESP-IDF, sets up a Python virtualenv, installs `xtensa-esp-elf`, `esp-clang`, `cmake`, `ninja`, and more. Expect this to take 10–20+ minutes and several GB of downloads. Subsequent builds are fast. + +### Known first-build failure points + +These are common enough to check for proactively rather than debug blind: + +**a) `esp-clang: error while loading shared libraries: libxml2.so.2`** Rolling-release Linux distros (Arch-based, etc.) ship a newer libxml2 soname than the pinned esp-clang binary expects. Fix: + +```bash +ldconfig -p | grep libxml2 # find your installed version, e.g. libxml2.so.16 +sudo ln -s /usr/lib/libxml2.so.16 /usr/lib/libxml2.so.2 # adjust path/version to match +sudo ldconfig +ls -la /usr/lib/libxml2.so.2 +``` +The final command should show a symlink of libxml2.so.2 to libxml2.so.16 + +**b) `Missing esp-mqtt submodule` / CMake `git_submodule_check` errors** The managed ESP-IDF checkout needs its git submodules initialized (several components are submodules even if your project doesn't use them): + +```bash +cd .embuild/espressif/esp-idf/v5.3.4 +git submodule update --init --recursive +cd - +cargo build --release +``` + +If that's still broken, delete and let it re-clone fresh: `rm -rf .embuild/espressif/esp-idf/v5.3.4`. + +A successful build ends with a normal `Compiling ` / `Finished release` output. + +--- + +## 8. Write a minimal ethernet-only main.rs + +This is the "ready for proper code" checkpoint — confirms SPI, the W5500, and DHCP all work before you build real functionality on top: + +```rust +use std::thread; +use std::time::Duration; + +use esp_idf_svc::eth::{BlockingEth, EspEth, EthDriver, SpiEthChipset}; +use esp_idf_svc::eventloop::EspSystemEventLoop; +use esp_idf_svc::hal::peripherals::Peripherals; +use esp_idf_svc::hal::prelude::*; +use esp_idf_svc::hal::spi::{config::DriverConfig, Dma, SpiDriver}; +use log::info; + +fn main() -> anyhow::Result<()> { + esp_idf_svc::sys::link_patches(); + esp_idf_svc::log::EspLogger::initialize_default(); + + let peripherals = Peripherals::take()?; + let sysloop = EspSystemEventLoop::take()?; + let pins = peripherals.pins; + + let spi_driver = SpiDriver::new( + peripherals.spi2, + pins.gpio13, // SCLK + pins.gpio11, // MOSI + Some(pins.gpio12), // MISO + &DriverConfig { + dma: Dma::Auto(4096), // required — default (no DMA) can't move full ethernet frames + ..Default::default() + }, + )?; + + let eth_driver = EthDriver::new_spi( + spi_driver, + pins.gpio10, // INT + Some(pins.gpio14), // CS + Some(pins.gpio9), // RST + SpiEthChipset::W5500, + 20.MHz().into(), + Some(&[0x02, 0x00, 0x00, 0x12, 0x34, 0x56]), // locally-administered MAC + None, // default PHY address + sysloop.clone(), + )?; + + let eth = EspEth::wrap(eth_driver)?; + let mut eth = BlockingEth::wrap(eth, sysloop)?; + + info!("Starting eth..."); + eth.start()?; + info!("Waiting for DHCP lease..."); + eth.wait_netif_up()?; + + let ip_info = eth.eth().netif().get_ip_info()?; + info!("Got IP: {:?}", ip_info); + + loop { + thread::sleep(Duration::from_secs(1)); + } +} +``` + +--- + +## 9. Flash and verify + +```bash +cargo install espflash # one-time, if not already installed +cargo build --release +espflash flash --monitor target/xtensa-esp32s3-espidf/release/ +``` + +If Linux user needs permission for port access: +```shell +sudo usermod -aG uucp $USER # Arch Systems + +sudo usermod -aG dialout $USER # Debian Systems +``` +Then reboot. + +Watch the serial monitor. Success looks like: + +``` +I (...) esp_idf_svc::eth: Driver initialized +I (...) hello_eth: Starting eth... +I (...) hello_eth: Waiting for DHCP lease... +I (...) hello_eth: Got IP: ... +``` + +If it hangs after "Waiting for DHCP lease..." with repeating `w5500.mac: w5500_spi_write failed` / `w5500_spi_read failed` errors in the log, the SPI bus doesn't have DMA enabled — double check the `Dma::Auto(4096)` line in step 8 is present. + +If it hangs with no errors at all and no IP, check physically: ethernet cable plugged into a live switch/router port, link LED on the RJ45 jack lit. + +--- + +## 10. You're ready to build on this + +At this point you have a project that: builds for the correct target, links the W5500 driver, brings up ethernet, and gets a DHCP lease. From here, typical next additions: + +- An HTTP server (`esp_idf_svc::http::server::EspHttpServer`) +- GPIO for LEDs/buttons/sensors (`esp_idf_svc::hal::gpio::PinDriver`) +- SPI/I2C peripherals sharing the bus alongside the W5500 (own CS pin, same or different SPI host) +- Async networking (`tokio`, `async-nats`, etc.) — needs additional rustflags and feature trimming; see the note on porting non-embedded async code if relevant to your project + +--- + +## Quick-reference: build error → fix + +|Error|Cause|Fix| +|---|---|---| +|`Unsupported target 'x86_64-unknown-linux-gnu'`|`.cargo/config.toml` target missing/wrong, or Xtensa toolchain not loaded in shell|Check `target = "xtensa-esp32s3-espidf"`; re-run `. $HOME/export-esp.sh`| +|`esp-clang: ... libxml2.so.2`|Host libxml2 soname mismatch|Symlink to your installed version (§7a)| +|`Missing esp-mqtt submodule` / CMake errors|Managed ESP-IDF checkout missing submodules|`git submodule update --init --recursive` in the IDF checkout (§7b)| +|`txdata transfer > host maximum` / w5500 read/write failures|SPI DMA disabled|`Dma::Auto(4096)` in `DriverConfig`| +|`EthDriver::new_spi` argument/type errors|API differs across esp-idf-svc versions|Match the compiler's reported signature exactly, not older examples/tutorials| +|No W5500 driver at runtime|`CONFIG_ETH_SPI_ETHERNET_W5500=y` missing from sdkconfig.defaults|Add it, rebuild| \ No newline at end of file diff --git a/Projects/ESP32-S3-ETH/Rust Dev Environment for Waveshare ESP32-S3-ETH (W5500).md b/Projects/ESP32-S3-ETH/Rust Dev Environment for Waveshare ESP32-S3-ETH (W5500).md deleted file mode 100644 index c927030..0000000 --- a/Projects/ESP32-S3-ETH/Rust Dev Environment for Waveshare ESP32-S3-ETH (W5500).md +++ /dev/null @@ -1,196 +0,0 @@ -Working setup, derived from an actual first-time bring-up. Every failure mode below was hit and fixed, in order. Follow this straight through and you skip all of it. (AI generated) - -## Board reference - -Waveshare ESP32-S3-ETH — ESP32-S3 + onboard W5500 ethernet chip over SPI (not RMII — there's no on-chip MAC/PHY involved, W5500 has its own MAC and does everything over SPI). - -Confirmed pin mapping (verify against your board's silkscreen — Waveshare pin assignments have varied slightly across production batches): - -|Signal|GPIO| -|---|---| -|SCLK|13| -|MOSI (SDO)|11| -|MISO (SDI)|12| -|CS|14| -|INT|10| -|RST|9| -|SPI host|SPI2| -|Clock|20–25 MHz| - -W5500 has no burned-in MAC address — you must supply one (a locally-administered address like `02:00:00:12:34:56` works fine). - -## Why std (esp-idf-svc), not no_std (esp-hal) - -`esp-generate` scaffolds `esp-hal` (`no_std`) projects. That toolchain has no W5500/ethernet driver and no full HTTP server story — its focus is Wi-Fi/BLE via `esp-radio` and `embassy`. For ethernet + a standard HTTP server, use the `std` toolchain via `esp-idf-svc`, which wraps ESP-IDF's C ethernet driver and httpd component. - -**Rule of thumb:** if the board has W5500/LAN8720/other wired-ethernet hardware, use `esp-idf-template` (std). Reach for `esp-generate` (no_std) only for pure Wi-Fi/BLE projects. - - - -## 1. Toolchain - -```bash -espup install -. $HOME/export-esp.sh # re-run in every new shell, or add to your shell rc -``` - -Confirm the Xtensa target is present: - -```bash -rustc --print target-list | grep xtensa -``` - - - -## 2. Scaffold the project - -Do **not** use `esp-generate`. Use: - -```bash -cargo install cargo-generate -cargo generate esp-rs/esp-idf-template cargo -``` - -Prompt answers that matter: - -|Prompt|Answer|Why| -|---|---|---| -|MCU|`esp32s3`|matches the board| -|Configure advanced options|Yes|need to reach STD toggle| -|STD support|**Yes**|pulls in `esp-idf-svc`; this is the whole point| -|ESP-IDF version|`v5.3.4`|most mature crates.io compatibility; avoid `master`/newest unless you have a reason| -|Use git esp-idf-* crates|**false**|crates.io releases are stable and documented; git HEAD drifts mid-project| -|Install location|**global**|ESP-IDF toolchain reused across projects, saves re-downloading (~1–2 GB)| -|Dev Containers|false|flashing real hardware over USB is simpler done natively| -|Wokwi simulation|false|Wokwi doesn't model W5500/ethernet peripherals| -|CI files|false|add later if/when you push to GitHub| - - - - -## 3. Known environment issues and fixes - -These hit in this order on a fresh Arch-based system (Omarchy); adjust distro-specific commands as needed, but expect similar issues on any non-Ubuntu-LTS host. - -### a) `esp-clang: error while loading shared libraries: libxml2.so.2` - -Rolling-release distros ship a newer libxml2 soname (`.so.16`+) than the pinned `esp-clang` toolchain expects (`.so.2`). Symlink it: - -```bash -ldconfig -p | grep libxml2 # find your installed version, e.g. libxml2.so.16 -sudo ln -s /usr/lib/libxml2.so.16 /usr/lib/libxml2.so.2 # adjust path/version to match -sudo ldconfig -``` - -Verify against the actual binary, not just `ldconfig -p` (which may not reflect symlinks immediately): - -```bash -/path/to/.embuild/espressif/tools/esp-clang/*/esp-clang/bin/clang --version -``` - -If a failed build already deleted the tool directory during its own error-recovery, just rebuild — `cargo build` reinstalls it. - -### b) `Missing esp-mqtt submodule` / CMake `git_submodule_check` errors - -The managed ESP-IDF checkout under `.embuild/espressif/esp-idf//` needs its git submodules initialized — several ESP-IDF components (mqtt, lwip, mbedtls, wifi/bt libs, etc.) are submodules, and IDF's build requires them present even if your project doesn't use those components. - -```bash -cd .embuild/espressif/esp-idf/ -git submodule update --init --recursive -cd - -cargo build --release -``` - -This is a one-time ~500MB–1GB download. If it's still broken afterward (partial/shallow clone issues), delete and let `embuild` re-clone fresh: - -```bash -rm -rf .embuild/espressif/esp-idf/ -cargo build --release -``` - -### c) Building for host instead of the chip (`Unsupported target 'x86_64-unknown-linux-gnu'`) - -`.cargo/config.toml` must set: - -```toml -[build] -target = "xtensa-esp32s3-espidf" -``` - -`esp-idf-template` generates this correctly — if you hit this, something overwrote or is missing that file, or `espup`'s toolchain isn't active in the current shell (`. $HOME/export-esp.sh`). - -### d) SPI DMA: `check_trans_valid: txdata transfer > host maximum` / w5500 read-write failures - -The default `SpiDriverConfig` has DMA disabled, which caps transfer size below what the W5500 ethernet driver needs to move full frames. Symptom: link never comes up, `wait_netif_up()` hangs, log fills with `w5500_spi_write failed` / `w5500_spi_read failed`. Fix — enable DMA explicitly when constructing the `SpiDriver`: - -```rust -use esp_idf_svc::hal::spi::{config::DriverConfig, Dma, SpiDriver}; - -let spi_driver = SpiDriver::new( - peripherals.spi2, - pins.gpio13, // SCLK - pins.gpio11, // MOSI - Some(pins.gpio12), // MISO - &DriverConfig { - dma: Dma::Auto(4096), - ..Default::default() - }, -)?; -``` - -### e) `esp-idf-svc` API drift across versions - -The `EthDriver::new_spi` signature has changed across `esp-idf-svc` releases — older versions took raw MOSI/MISO/SCLK pins directly; 0.51+ takes a pre-built `SpiDriver` plus INT/CS/RST only (9 args total: `spi, int, cs, rst, chipset, baudrate, mac_addr, phy_addr, sysloop`). If code from an older tutorial/example doesn't compile, check the exact signature for your pinned version: - -```bash -cargo doc --open -p esp-idf-svc -# or browse https://docs.rs/esp-idf-svc//esp_idf_svc/eth/ -``` - -Trust the compiler's error over memory or old examples — it names the exact expected type/arg count. - - - - -## 4. sdkconfig.defaults - -``` -CONFIG_ETH_SPI_ETHERNET_W5500=y -CONFIG_ESP_MAIN_TASK_STACK_SIZE=8000 -CONFIG_HTTPD_MAX_REQ_HDR_LEN=1024 -``` - -Without the first line, the W5500 driver component isn't built into ESP-IDF at all — you'll get a runtime or link failure referencing missing eth symbols. - - - -## 5. Cargo.toml essentials - -```toml -[dependencies] -esp-idf-svc = { version = "0.51", features = ["std"] } -anyhow = "1" -log = "0.4" -``` - - - -## 6. Build / flash / monitor - -```bash -cargo build --release -espflash flash --monitor target/xtensa-esp32s3-espidf/release/ -``` - -Watch the serial log for `Got IP: ...` — that confirms both SPI-to-W5500 link and DHCP succeeded. No `Got IP` line + streaming `w5500.mac` errors = revisit DMA config or pin mapping (item d/a above). - -## Summary: what each fix actually solved - -|Problem|Root cause|Fix| -|---|---|---| -|Config menu had no ethernet option|Wrong scaffolding tool (`esp-generate` = no_std/esp-hal)|Use `esp-idf-template` (std) instead| -|Build targeted host, not chip|Missing/wrong `.cargo/config.toml` target|Ensure `target = "xtensa-esp32s3-espidf"`| -|`esp-clang` failed to load|Host libxml2 soname too new|Symlink `.so.16` → `.so.2`| -|CMake submodule errors|Managed ESP-IDF git checkout incomplete|`git submodule update --init --recursive`| -|`EthDriver::new_spi` type errors|API changed across esp-idf-svc versions|Match compiler-reported signature exactly, don't trust memorized old examples| -|W5500 never links / SPI transfer errors|DMA disabled by default on SPI bus|`Dma::Auto(4096)` in `DriverConfig`| \ No newline at end of file