10 posts / 0 new
Last post
mabraun
Offline
Last seen:3 years 6 months ago
Joined:2015-11-16 15:57
SPI GTL Deadlock

Hi all,

I'm operating a DA14580 in the external processor configuration. It is controlled by a uC (SPI master) using the GTL 5-wire SPI protocol. I keep having the problem of running into a deadlock. When streaming some data over BLE (hence a lot of traffic between the uC and the DA), I run into a state where both the DA and the uC want to transmit at the same time. More specifically, the DA gets stuck in thespi_hci_write_func()function. It drives the DREADY signal high and waits for an SPI transfer (spi_wait_for_transfer()):

...
spi_dready_high(); // Assert dready to request transmit
do {
spi_wait_for_transfer(); // Wait for SPI transaction from SPI master device
...

However, the uC tries to send a message at the same time and gets stuck in thespi_send_hci_msg()function, waiting for the DREADY signal to go low:

...
// disable dready interrupt
dready_irq_disable();
// Polling DREADY to detect if data is being received
while(dready_get_status());
...

The following happens:
- The uC disables the dready interrupt, just before the DA asserts the dready signal.
- The dready signal goes high, but no interrupt is triggered on the uC side.
- Instead, the uC waits for the dready signal to go low.
- The dready signal won't go low, since the DA is waiting for an SPI transmission.
--> Deadlock

I'm wondering how the protocol intends to avoid running into the deadlock described above. From what it looks like there is no mechanism that prevents that situation.

Thanks!
mabraun

edit: tried to make problem description more comprehensive

Keywords:
MT_dialog
Offline
Last seen:2 months 1 week ago
Staff
Joined:2015-06-08 11:34
Hi mabraun,

Hi mabraun,

There is a sequence in transfering data over the SPI bus the signals and the data exchanged by the devices are not only the DREADY in order to prevent this condition, there are flow_on and flow_off data that should be exchanged before the two devices can transfer data over the SPI bus. You can have a look at the UM-B-013 in order to check the protocol in detail. There is a specific procedure before the master can talk to slave and vice versa in order to avoid this kind of deadlock, before the master can transmit should check if the slave is available (if the last message send from slave was a flow on byte) before invoking the spi_send_hci_msg() and disabling the interrupts.

Thanks MT_dialog

mabraun
Offline
Last seen:3 years 6 months ago
Joined:2015-11-16 15:57
Hi MT_dialog,

Hi MT_dialog,

Thanks for your reply. I am well aware of the flow_on/flow_off principle, I have read through UM-B-013 carefully. It is worth clarifying that I used the source code provided in the proximity reporter example from SDK5 (for both the host and the DA), which is why the flow control mechanism is (or should be) properly implemented. The problem I'm having is not related to the flow control mechanism and is even addressed in section 6.2.3 of UM-B-013:

To prevent the event of simultaneous transmissions from both master and slave(i.e. the master
sends the 0x05 start byte to begin the transmission of a message and the slave at the same time
sends a flow off byte to subsequently transmit another message)
,主acknowledge the
DREADY request by the slave.

The highlighted example is exactly the problem I'm experiencing: master wants to send message (0x05), slave wants to send flow off byte. It continues:

To do so, when the master detects DREADY to be active, it sends an
acknowledgement byte (ACK, chosen to be 0x08) to inform the slave that it can go on to send the
flow off byte (or any other data). This functionality is shown in Figure 3. On any DREADY rising edge,
the master has to acknowledge that it detected it and enable the slave device to send data.If the
master does not acknowledge the DREADY rising edge, the slave waits until the Chip Select line is
inactive (which will indicate that the master has finished sending the message), activate DREADY
again, and wait for the acknowledgement to proceed to transmit.

However, the behaviour described above (bold) doesn't seem to be implemented on the DA side (see spi_hci.c):

bool spi_hci_flow_off_func(void)
{
uint8_t tmp;
// First check if no transmission is ongoing
if((spi_cs_getf()==0))
{
return false;
}
NVIC_DisableIRQ(SPI_IRQn); // Disable SPI interrupt to CPU
spi_dready_high(); // Assert dready to request transmit
do {
spi_wait_for_transfer(); // Wait for SPI transaction from SPI master device
tmp = spi_rxtxreg_read(); // Get byte from SPI
} while (tmp != DREADY_ACK); // If DREADY is not acknowledged, try again
[...]
}

The code above first checks if a transmission is ongoing,THENit asserts the DREADY line,thenit waits for an SPI transfer (or the DREADY_ACK byte) indefinitely. This behaviour doesn't correspond to:

If the master does not acknowledge the DREADY rising edge, the slave waits until the Chip Select line is inactive (which will indicate that the master has finished sending the message), activate DREADY again, and wait for the acknowledgement to proceed to transmit.

Let's look at the uC (host) side: Section 6.2.3 also states that

On any DREADY rising edge, the master has to acknowledge that it detected it and enable the slave device to send data.

The code from spi_hci_msg.c looks like this:

空白spi_send_hci_msg (uint16_t大小,uint8_t * msg_ptr)
{
uint16_t i;
// disable dready interrupt
NVIC_DisableIRQ(GPIO0_IRQn);
// Polling DREADY to detect if data is being received
while(GPIO_GetPinStatus(SPI_GPIO_PORT, SPI_DREADY_PIN));
spi_cs_high(); // Close CS
spi_cs_low(); // Open CS
spi_access(0x05);
[...]
}

The codeFIRST禁用所打断,thenit polls for DREADY,thenit pulls the SPI chip select line low. If DREADY goes high right after the interrupt has been disabled, the uC will wait indefinitely for DREADY to go low again and the DREADY irq request will not be serviced. Also, the CS line does not prevent this from happening, as it is pulled low after deactivating the DREADY interrupt.

Am I missing something here?

Sorry for the long post, but I want to make sure I'm being understood correctly.

Cheers!

MT_dialog
Offline
Last seen:2 months 1 week ago
Staff
Joined:2015-06-08 11:34
Hi mabraun,

Hi mabraun,

Please excuse me in case i got something wrong, you ve noticed that when there is an ongoing transaction from the master, so the slave has send a flow on, and the master has send a 0x08 in order to acknowledge it, so the master is ready to send data and disables the IRQ and after that immidiatelly the slave issues a flow_off in order to send data, so the DREADY pin gets high at the time that the CS is disabled (is high so it evades the CS check in the spi_cs_getf() function) since the DREADY is pulled high the master is stuck in the polling procedure and the slave is stuck waiting for a 0x08 message that will never come from the master.

Have you experienced this kind of issue in your application and if you did, do you have a capture of the SPI transactions ?

Thanks MT_dialog

mabraun
Offline
Last seen:3 years 6 months ago
Joined:2015-11-16 15:57
Hi MT_dialog,

Hi MT_dialog,

yes, you got it exactly right. In the attached image you'll find 4 signals:
1. SPI CLK (yellow)
2. SPI MISO (blue)
3. SPI CS (pink)
4. DREADY (green)
My scope only has 4 channels, so I couldn't capture the MOSI line.

In the beginning of the screenshot you can see the flow_off communication: 2 SPI bytes, containing the DREADY_ACK and the FLOW_ON bytes. After that, the master initiates a transmission (notice how the CS line goes low while DREADY is inactive). After that (successful) transmission, the slave asserts the DREADY signal (because it wants to send a flow_off message), and the master wants to send a message as well and pulls the CS line low. (for better visualization I moved thespi_cs_low()code line beforewhile(dready_get_status());. Otherwise you wouldn't see the CS line being pulled low.) Notice how both DREADY and CS are asserted (CS low, DREADY high) at almost the same time, but no subsequent SPI communication is visible. That is when the deadlock happens.

Thanks for your help!
mabraun

Attachment:
MT_dialog
Offline
Last seen:2 months 1 week ago
Staff
Joined:2015-06-08 11:34
Hi mabraun,

Hi mabraun,

I ve checked with the support team and as far as they could tell me is, this scenario is likely to happen, there is no timeout or any other mechanism that would prevent the 580 from sending a flow off command while the host just started sending data and that the deadlock seems that it can occur, rarely but its possible. Because we ve never ecountered this kind of issue, when operating over SPI can you make sure that this deadlock is for sure the reason for the stalling of your system, can you provide a more complete capture of your signals, in order to make sure that the 0x05 is left from the host (MOSI signal) or if you are able to debug, have you confirmed that this is where the code stucks in both sides (spi_send_hci_msg() and spi_hci_flow_of_func() ) ?

Thanks MT_dialog

mabraun
Offline
Last seen:3 years 6 months ago
Joined:2015-11-16 15:57
Hi MT_dialog,

Hi MT_dialog,

thanks for your reply. In my current setup I am able to debug both sides, which is why I know that both sides are indeed stuck in spi_send_hci_msg() and spi_hci_flow_off_func().

I attached 4 more screen captures. I captured all 5 signal lines (all displayed in purple), in the following order: CLK, MOSI, MISO, CS, DREADY. Additionally, I used a protocol analyzer to interpret the MOSI and MISO lines, shown in blue on the screenshots. Bus 1 corresponds to MOSI, Bus2 to MISO.

1.png:Overall view on the problematic sequence resulting in the deadlock.

2.png:Zoomed-in view of the Flow-On sequence. DREADY is high, Master acknowledges it (0x08) and Slave sends FlowOn byte (0x06).

3.png:Zoomed-in view of the first byte sent by the Master, marking the beginning of a message (0x05).

4.png:放大视图发送的消息的结束the Master (CS is pulled high). Afterwards, the deadlock occurs: Slave wants to send FlowOff (which begins by asserting DREADY), Master wants to send the next message. Please note: In contrary to my previous post, I didnotmove thespi_cs_low()code line beforewhile(dready_get_status()), which is why you cannot see the CS line being pulled low. This is the original code behaviour. The problem, however, remains the same: The master is stuck in spi_send_hci_msg(), waiting for the DREADY signal to go low. The slave is stuck in spi_hci_flow_off_func(), waiting for an SPI transfer.

Please let me know if you need any additional information.

Thanks for your help,
mabraun

Attachment:
mabraun
Offline
Last seen:3 years 6 months ago
Joined:2015-11-16 15:57
Any updates on this? Will you

Any updates on this? Will you be able to provide a fixed implementation of the 5-wire SPI protocol?

Thanks,
mabraun

MT_dialog
Offline
Last seen:2 months 1 week ago
Staff
Joined:2015-06-08 11:34
Hi mabraun,

Hi mabraun,

At the time being the issue is filed as a change request, so far we have a possible solution in order to overcome this deadlock but its going to take some time in order to test it and evaluate any issues or limitations. I will have more about this withing this or next week, i will let you know.

We are sorry for any inconvenience.

Thanks MT_dialog

mabraun
Offline
Last seen:3 years 6 months ago
Joined:2015-11-16 15:57
Hi MT_dialog,

Hi MT_dialog,

thanks for your update, that is great news. Please keep me posted.

Regards,
mabraun