Scoutinternal

XLM (Stellar) Transaction Issues: Stuck in Signed Status, Missing Deposits, and Failed Withdrawals

XLM (Stellar) Transaction Issues: Stuck in Signed Status, Missing Deposits, and Failed Withdrawals

Problem

Customers using BitGo XLM (Stellar) wallets frequently report outgoing transactions stuck in "signed" status that do not broadcast to the Stellar network for extended periods (30 minutes to 12+ hours), incoming deposits confirmed on-chain but not appearing in BitGo wallet balances, and withdrawal transactions failing with various Stellar network errors. These issues predominantly stem from BitGo's XLM indexer falling behind the chain head, full node connectivity problems (HTTP 503), sequence ID mismatches (tx_bad_seq), missing memo type fields in API payloads, and wallet initialization requirements. The problems affect both mainnet (XLM) and testnet (TXLM) wallets.

Diagnostics

  • Check transaction status with bga t <txid>: Verify whether the transaction is in the block chain, in the send queue (and its state: done, attempted, failed), and whether wallet transfer records exist. Compare the BitGo status against a public explorer (stellarchain.io, stellar.expert, blockchair.com).
  • Inspect the send queue error message: Look for specific errors such as:
    • Failed to connect to full node with HTTP 503 and "Historical DB Is Too Stale" — indicates the Horizon node is behind.
    • tx_bad_seq — indicates a sequence number mismatch on the Stellar network.
    • Transaction Malformed with empty envelope_xdr — indicates a stuck indexer that corrupted the transaction envelope.
    • wallet pending on-chain initialization — the wallet's pendingChainInitialization flag is still true.
  • Check bga w <walletId>: Confirm wallet balance, spendable balance, and whether "pendingChainInitialization": true appears in the JSON output.
  • Compare BitGo wallet balance to on-chain balance: Use a Stellar explorer to verify the root address balance matches what BitGo reports. A discrepancy suggests missed indexing.
  • Check the BitGo status page: Visit https://status.bitgo.com/ for any active incidents related to XLM/Stellar.
  • Check wallet transfer records for no wallet tx records: If bga t <txid> shows the transaction confirmed on-chain but with ✘ no wallet tx records, the transaction was missed by the indexer and needs rebroadcast/reindexing.
  • For API errors, inspect the payload: Verify the memo object includes both type and value fields. Check the BitGo Express / SDK version in use.
  • For testnet issues: Check https://status.stellar.org/ for quarterly testnet resets, which invalidate existing TXLM wallets.

Resolution

Scenario: xlm-transaction-signed-transactions#indexer-delay-stuck-signed

Trigger: Outgoing XLM transaction shows "signed" status in BitGo but is either not yet broadcast or already confirmed on-chain, typically accompanied by indexer lag or full node connectivity errors (HTTP 503).

Signals: signed, stuck, outgoing, XLM, indexer, 503, stale_history, Historical DB Is Too Stale, Failed to connect to full node, delay, broadcast

Steps:

  1. Run bga t <txid> to check the transaction state. Look for errors in the send queue such as Got 503 or Historical DB Is Too Stale.
  2. Verify on a public Stellar explorer whether the transaction has actually confirmed on-chain.
  3. If the transaction is confirmed on-chain but BitGo still shows "signed", escalate to the Engineering team (via the eth-alt Slack channel) to investigate and fix the XLM indexer.
  4. If the transaction is stuck in the send queue with repeated broadcast failures due to node connectivity (503), escalate to DevOps to check the XLM Horizon node health.
  5. Once engineering resolves the indexer issue, confirm the transaction status has updated to "confirmed" in BitGo and inform the customer.
  6. If some transactions ended up in "failed" state due to the indexer being stuck for too long, advise the customer to re-initiate those transactions.

Notes: This is a recurring issue. Multiple incidents have been documented where the XLM indexer falls behind the chain head, causing delays of 30 minutes to 12+ hours. Engineering has implemented fixes multiple times but the issue can recur. Direct the customer to https://status.bitgo.com/ for real-time updates during active incidents.

"For these specific 2 transactions, there was an issue in reaching the full node that caused them to get stuck, but our engineering team fixed the issue and the transactions were pushed to chain and got confirmed under the bitgo wallet." "we had some downtime on XLM indexers due to some internal technical issue which has caused delays in XLM transactions about 45 min to 1 hour. They have also informed that they have temporarily fixed the issue and likely having some slowness in XLM transactions until they permanently fix the issue." "We have fixed an issue on our xlm-indexer and all XLM transactions should be confirmed now. Please let us know if you still see any stuck transaction."

Scenario: xlm-transaction-signed-transactions#incoming-deposit-not-appearing

Trigger: Incoming XLM deposit is confirmed on the Stellar blockchain but does not appear in the BitGo wallet transfer history.

Signals: deposit, receive, not found, missing, incoming, no wallet tx records, reindex, rebroadcast

Steps:

  1. Run bga t <txid> and confirm the transaction shows ✓ in block chain but ✘ no wallet tx records.
  2. Run bga indexer rebroadcast <txid> to reindex the transaction.
  3. Run bga t <txid> again to verify the transaction now shows ✓ 1 wallet transfer records with status confirmed.
  4. Inform the customer that the deposit should now be reflected in their BitGo wallet.

Notes: This typically happens when the BitGo indexer misses an incoming transaction. Multiple rebroadcasts for different transactions may be needed if several deposits were missed during an indexer disruption.

"We have reindexed the following transactions and it should now be confirmed on your side." "We have reindexed this transaction and it is now confirmed. ... ✓ wallet transfer records: ✓ wallet:[WALLET_ID] transfer:[WALLET_ID] 103.98 confirmed"

Scenario: xlm-transaction-signed-transactions#status-mismatch-on-chain

Trigger: XLM transaction status in BitGo does not match the on-chain status — e.g., BitGo shows "signed" but the transaction succeeded on-chain, or BitGo shows "signed" but the transaction failed on-chain.

Signals: wrong status, signed, confirmed on chain, failed on chain, status mismatch

Steps:

  1. Run bga t <txid> and compare the BitGo transfer state against the on-chain state via a Stellar explorer.
  2. If the transaction is confirmed on-chain but BitGo shows "signed" or another incorrect status, escalate to the Engineering team to manually update the transfer state.
  3. If the transaction failed on-chain but BitGo shows "signed", escalate to Engineering. Once the status is corrected to "failed", advise the customer to re-initiate the transaction.
  4. Follow up with the customer once Engineering confirms the status has been corrected.

Notes: Engineering typically needs to update the transfer record directly. This scenario often co-occurs with indexer issues.

"Status for this transaction with hash d8719b... was resolved by our engineering team. And for this one 6a992... our team is still working on it."

Scenario: xlm-transaction-signed-transactions#tx-bad-seq

Trigger: XLM transaction fails with Stellar network error tx_bad_seq indicating a sequence number mismatch.

Signals: tx_bad_seq, bad sequence, sequence number, failed, Transaction Failed

Steps:

  1. Run bga t <txid> and check the error details for "result_codes":{"transaction":"tx_bad_seq"}.
  2. Explain to the customer that tx_bad_seq means an incorrect sequence number was used. This can happen when:
    • Multiple transactions are submitted in rapid succession from the same account.
    • A previous transaction in the sequence has not yet confirmed, causing subsequent ones to fail.
  3. If the sequence ID was too high, the transaction may still confirm once earlier sequence IDs are processed. Monitor and wait.
  4. If the transaction has permanently failed, advise the customer to re-initiate the withdrawal.
  5. Ask the customer whether transactions are created via UI or API. If via API, request: full endpoint, full payload (omitting wallet passphrase), email of the user who created the access token, and BitGo Express/SDK version.

Notes: In situations where the sequence ID used was too high, the transaction may still confirm if still valid when its sequence ID is reached in processing order. Rapid successive sends from the same wallet are the most common trigger.

"The transaction failed due to error: tx_bad_seq which represents that an incorrect sequence number was used. This could be a result of using one that was already used or one that is too high and as a result, is out of sequence."

Scenario: xlm-transaction-signed-transactions#invalid-memo-missing-type

Trigger: XLM withdrawal fails with error "invalid memo undefined: <value>" or "invalid memo memoId: <value>" when sending via the API.

Signals: invalid memo, memo, undefined, type, sendcoins, API, withdrawal failed

Steps:

  1. Review the customer's API payload for the memo object. Confirm it includes both type and value fields.
  2. If the type field is missing, instruct the customer to add it. Valid values for the memo type field are id or text.
  3. Ensure the trustlines and memo parameters are provided as arrays where required per the API documentation at https://developers.bitgo.com/api/express.wallet.sendcoins.
  4. Ask the customer to retry the withdrawal with the corrected payload.

Notes: The memo object must include both "type" and "value". Omitting "type" produces "invalid memo undefined: <value>". Using an incorrect type string (e.g., "memoId" instead of "id") also produces an error.

"Reviewing this error, it appears the field 'Type' was left out of the payload with regards to the memo. ... In speaking with our engineering team, they advise id or text should be the a valid string value for 'Type'."

Scenario: xlm-transaction-signed-transactions#pending-chain-initialization

Trigger: XLM wallet returns error "wallet pending on-chain initialization" when attempting to send, even though the wallet has an on-chain balance.

Signals: pending on-chain initialization, pendingChainInitialization, wallet creation, initialization, 2.6 XLM

Steps:

  1. Run bga -j w <walletId> and check if "pendingChainInitialization": true appears in the output.
  2. If the wallet has no on-chain balance, advise the customer that XLM wallets require approximately 2.6 XLM total sent to the wallet address to initialize. A minimum of 1 XLM is required to activate, and 0.5 XLM must be maintained for the wallet to stay functional.
  3. If the wallet already has sufficient on-chain balance but pendingChainInitialization is still true, this is a bug. Escalate to the Engineering team to manually fix the wallet's initialization flag.
  4. Confirm with the customer that sends work after the fix.

Notes: After wallet creation, the UI shows "Status Pending Chain Initialization" until the wallet is funded. For new wallets, approximately 2.6 XLM total is needed. If the wallet already has balance and this flag persists, it requires an engineering fix. It is advisable to check other XLM wallets under the same enterprise for the same issue.

"For XLM wallets, they require a 1 XLM transaction sent to them to activate the wallet and require a 0.5 XLM balance for the wallet to stay functional/activated. ... To initialize these wallets, they need roughly 2.6 XLM total sent to the wallet." "We have fixed this XLM wallet and you should no longer be seeing the mentioned error."

Scenario: xlm-transaction-signed-transactions#muxed-address-prebuild-mismatch

Trigger: XLM withdrawal to a Muxed address (M-prefixed) fails with error "transaction prebuild does not match expected recipient".

Signals: muxed, Muxed Account, prebuild, does not match expected recipient, M-address

Steps:

  1. Confirm the customer is sending to a Muxed address (starts with M).
  2. Verify the BitGo Express / SDK version in use.
  3. This was a known bug fixed in BitGoJS v23.0.0 and Express v9.46.0. Instruct the customer to update to at least these versions.
  4. After updating, the customer should retry the withdrawal.

Notes: BitGo does support Muxed Accounts. The fix was specifically deployed in BitGoJS v23.0.0 and Express v9.46.0.

"We would like to inform that this is fixed in the latest BitGoJS v23.0.0 & Express v9.46.0. Please update your SDK and then retry this withdrawal and it should work."

Scenario: xlm-transaction-signed-transactions#webhook-not-received

Trigger: Customer reports not receiving a webhook notification for a confirmed XLM transaction.

Signals: webhook, not receive, notification, simulate, missing webhook

Steps:

  1. Obtain the wallet ID and transaction hash from the customer.
  2. Run bga wallet webhooks and bga wallet webhooknotifications to verify the webhook configuration and check for failures.
  3. If no notification was generated for the specific transaction and no webhook failures are logged, advise the customer to use the Simulate Webhook API endpoint to trigger the notification manually: https://developers.bitgo.com/api/v2.wallet.simulatewebhook
  4. If the webhook shows failingSince or successiveFailedAttempts > 0, investigate the customer's receiving endpoint.

Notes: Occasionally webhook notifications are not generated for individual transactions even when the webhook is active and healthy. The simulate endpoint is the standard workaround.

"We are not showing a notification for this transaction. Your team can simulate this webhook notice using the following API endpoint: https://developers.bitgo.com/api/v2.wallet.simulatewebhook"

Scenario: xlm-transaction-signed-transactions#xlm-trustline-api

Trigger: Customer needs to add a trustline for a Stellar token (e.g., USDC) to their XLM wallet via API and receives errors such as "Missing parameter: address" or "invalid trustline action: undefined".

Signals: trustline, token, USDC, XLM, add trustline, Stellar Tokens, Missing parameter address, invalid trustline action

Steps:

  1. Direct the customer to the correct API endpoint for initiating a trustline transaction: https://developers.bitgo.com/api/v2.wallet.trustline.initiate
  2. Ensure the trustlines parameter is provided as an array of objects, each containing token, action, and limit.
  3. For qualified custody wallets, customers should email support@bitgo.com to have BitGo create the trustline on their behalf.
  4. If the customer encounters a 500 error when calling /tx/send with a half-signed trustline transaction, check for TypeError: Cannot read properties of undefined (reading 'switch') in logs — this typically indicates the trustlines were not passed as an array.

Notes: The documentation at https://developers.bitgo.com/coins/Stellar%20Tokens has been reported as having broken links. The trustline initiate endpoint is the recommended path.

"I believe you will want to use the following endpoint for Initiating a Trustline Transaction: https://developers.bitgo.com/api/v2.wallet.trustline.initiate"

Scenario: xlm-transaction-signed-transactions#testnet-reset

Trigger: TXLM (testnet XLM) wallets show as corrupted, deleted, or transactions are permanently pending after a Stellar testnet quarterly reset.

Signals: testnet, TXLM, corrupted, deleted, quarterly reset, Pending Chain Initialization, testnet reset

Steps:

  1. Check https://status.stellar.org/ for recent testnet reset announcements. Stellar resets their testnet approximately once per quarter.
  2. Explain to the customer that after a testnet reset, all existing TXLM wallets are invalidated and must be recreated.
  3. If deleted wallets are still appearing in the BitGo UI, escalate to Engineering to remove them from view (this is a known UI bug).
  4. Advise the customer to create new TXLM wallets for testing.

Notes: Testnet resets invalidate all existing wallets. BitGo needs to reset the TXLM indexer after each Stellar testnet reset, which may take a couple of days. See https://www.stellar.org/developers/guides/concepts/test-net.html#periodic-reset-of-testnet-data

"XLM testnet is currently usable in BitGo as there is a XLM testnet reset on 11th June 2024 (https://status.stellar.org/incidents/v8pp13s1cw4d). Testnet resets typically happen once per quarter."

Scenario: xlm-transaction-signed-transactions#balance-mismatch-after-indexer-fix

Trigger: XLM wallet balance in BitGo does not match the on-chain balance, typically following an indexer issue or maintenance.

Signals: balance mismatch, incorrect balance, wallet balance, chain balance, out of sync

Steps:

  1. Run bga w <walletId> and compare the reported Balance against the on-chain balance via a Stellar explorer (e.g., stellarscan.io).
  2. If there is a discrepancy, escalate to Engineering to reindex missing transactions or resync the wallet balance.
  3. Once reindexing is complete, verify the balance matches (note: there may be a ~2.5 XLM difference due to the wallet initialization reserve).
  4. Confirm with the customer that balances are now in sync.

Notes: A difference of approximately 2.5 XLM between the BitGo spendable balance and on-chain balance is expected due to the Stellar base reserve for wallet initialization.

"Wallet balance should be in sync with chain balance by now. ... There would be a difference of 2.5xxx XLM for wallet initialisation"

Scenario: xlm-transaction-signed-transactions#nonce-hole-create-account-race

Trigger: XLM transactions fail due to a nonce hole caused by rapid successive sends, particularly when a createAccount transaction races with a payment transaction to the same new address.

Signals: nonce hole, createAccount, create account, rapid sends, race condition, failed

Steps:

  1. Check the transfer history and error details. Look for patterns of multiple transactions to the same new (uninitialized) address in quick succession.
  2. Explain to the customer that when a createAccount transaction has not yet been broadcast/confirmed, another createAccount transaction may be created instead of a payment transaction, causing failures.
  3. Engineering has implemented a fix that checks the send queue before building a create account transaction and throws an error advising the user to wait. Confirm the customer is on a current SDK version.
  4. If the failed transaction is due to a nonce hole that has since been resolved, advise the customer to re-initiate the transaction.

Notes: This primarily affects scenarios where automated systems send multiple transactions very quickly to new addresses. Subsequent successful transactions from the same wallet indicate the nonce hole has been resolved.

"We see this transfer is from 2 weeks ago and failed due to a nonce hole which is already fixed. We also see there were successful transfers from this wallet after this one."

Related