Is there not a timing issue worth talking about?

[daveythacher]

The way you program the flash creates pauses between Intel hex lines. This means that the sender should stop for some amount of time or be sent a notification to send the rest. This is technically application specific and would need to be provide in app.c, however this is not currently mentioned.

Could you mention this?

[daveythacher]

This avoided with SDK and 64KB max hex? Consuming lower 192KB of flash.

[rhulme]

I'm not sure exactly what you mean. The test application reads the entire image and then flashes it in one operation. There's no requirement for the sender to pause between between lines.

The test application will only work reliably with images of less than 64kb because the Intel hex format only specifies a 16-bit offset in each line. For a larger address space, the 'extended segment address' or 'extended linear address' records would have be to processed (or processed more accurately). The test application is only for demo purposes and easily fits in 64kb so there was no need to make it more complicated.

The flashloader doesn't care where the new image is stored. It can be anywhere in flash as long as it starts on a 4k boundary and there is no overlap between where the flashloader reads it from and where it gets written to.

How a new image is received is outside the scope of this project because there are any number of ways to do it and most will be specific to the individual project's requirements. The test app just gives a very simple way of testing the flashloader functionality.

[daveythacher]

I was mostly saying that there is some confusion based on the readme about how to get this working. This is a little abnormal from what I am used too. It was unclear to me how you supported 16MB of flash. It was unclear to me how you handled page erase time over UART. It was unclear to me how you prevent overwriting the loader. It was unclear to me what format or protocol you supported.

I was mostly curious to see how hard it would be port this to RS-485. It should be possible however I may need to rework some of this. It was meant to be constructive feedback. I did not want to create an issue for it since it would be better to ask first.

[rhulme]

The discussion is fine, I just couldn't quite understand the question.

Each line received via UART is added to the end of a RAM buffer. The contents of the buffer are only written to flash once everything has been received so there is no handshaking necessary to tell the sending device to wait. The code is designed to work with the hex files generated by the build process.

An "extended linear address" record is taken as the start of the data and the current write offset within the buffer is set to zero.
For each data line that is received, the payload is copied to the end of the buffer and the write offset is incremented accordingly.
When an "end of file" record is received, the contents of the buffer is written to flash (erase space, then write).

That is all done in readIntelHex(). which in turn calls flashImage() once.

The received image is written to the address specified in FLASH_IMAGE_OFFSET, which happens to be at the 128k mark but there is nothing magical about that number. It can be anywhere in flash. The flashloader will search through the whole of the available flash until it finds a new image (see

pico-flashloader/flashloader.c

Line 337 in 526fbfa

while(image < (XIP_BASE + PICO_FLASH_SIZE_BYTES))

).

The only restriction on where the new image can be stored is that it must be far enough away from the end of the application when it is flashed that the two do not overlap. So the flashloader will write the new image starting at FLASH_START + 4k. Let's say the existing application fits in 8k, and it writes the image it receives to FLASH_START + 4k + 8k, but the new application is bigger and needs 9k. If the flashloader were to blindly erase 9k (which would be rounded up to 12k because each erase block is 4k), it would erase the beginning of the new image before it had been copied.

So you either pick a fixed address far away from the active application, or you pick an address dynamically. Either way, there is nothing in the flashloader that expects an image at a certain address.

How the new image is transferred in is entirely up to you and is not part of this project. The same goes for how that data is written to flash before triggering the flashloader. In the demo app, everything is written in one go from a RAM buffer. For a large image, that might not be feasible. Equally, the time taken for erasing and flashing might be a concern for the other operations of whatever your application does, so you might need to erase and flash in chunks to prevent timing problems there.

You can use any interface available to you (whether UART, SPI, i2c, USB or something implemented via GPIOs and/or PIO) and the format of the data is also up to you. A text format like Intel Hex is a bit bulky but that might not be an issue. If you only have a very low bandwidth connection available, you'd probably want to look at using some form of compression on the raw binary data. The only requirement from the flashloader's point of view is that the image it finds has the correct header (see tFlashHeader) followed by the exact sequence of binary data to be flashed to the final location.

[daveythacher]

What happens if the malloc fails? According to the current logic you would need to hold this amount of memory. It is more common to not use a library for this very reason. I am guessing malloc is used to avoid wasting stack in application. 64K is a huge allocation compared to 128, 512, 1024, 4096, etc.

The max binary size is something like 256K before you are out of RAM. (Size should likely be a macro.) The max flash image size is 8MB, but the current logic does not support this. In order to support 8MB you would need to transfer multiple 64KB blocks. Currently this is not implemented but could be added. In this case you need to pause IO.

Speaking of flash range. It is important to know where this boundry is as some devices use the flash for filesystems. These would need to be known in the linker and/or application logic.

Note I could probably hack the application logic. Send a command to reboot the application and wake up in special mode. In this case I have entire heap which would need to be set something larger than 64KB. This would limit my stack, but if I know about it, I can work around it. If I want to use less than 64KB blocks, I just need to know how to do it. This will allow me to work around the stack limits.

One alternative you could use is a RAM function loader, to avoid allocating two copies in flash. I do not recommend this approach. The library approach is not standard. It is preferred to use a reserved section of memory and make it independent from the application. That being said I can think of a few reasons for why you would want to potentially kick your IO loader. Change IO standard, remove it, control start up time, rework memory layouts, etc.

Currently you IO loader is part of the application as a library. If this is ever dropped or broken the device is bricked. Adding more stages is wasteful and complicated. That being said it is a nice option to have if you mess up. Again, I could probably hack this in with application logic.

[rhulme]

As I wrote in the README (with some additional emphasis here):

The demo application included here is just to show how everything works. It is not designed to be incredibly robust.

Yes, using a 64k RAM buffer in this way limits the application size to less than 64k.
Yes, malloc() might fail.

But it's a demo application! It's there to demonstrate the overall principle and to show that the flashloader works. I could have read the data into a 4kb buffer and flashed in chunks, but then I would have had to deal with flow control, which would have added more code unrelated to the actual flashloader and meant that anyone testing would also need a setup that supported flow control.

The "application" itself just flashes an LED and needs next to no RAM to do so, so allocating 64k for incoming data is not a problem. In a real-world application that may not be the case so you'd need a different approach. But then again, maybe it wouldn't be a problem so why not allocate a large buffer if it's possible to do so and it maybe removes the need for flow control? Whether the buffer is statically allocated or dynamically allocated is neither here nor there as long as it works.

If your application is too large to completely fit into a RAM buffer then of course you'll have to write it in blocks. The flashloader doesn't care how the data came in or how it was written. You can throw out everything in app.c and the flashloader will still work the same. It has absolutely no dependency on the code in app.c.

If your application is more than half the size of the available flash, then the approach taken here is obviously not going to work as-is. The flashloader could, of course, be changed so that it receives the new application image and writes it directly to the final location. That adds more complexity to the flashloader, meaning it's more likely that you will want to be able to update the flashloader too, so you would need to add an additional loader before it in the chain that would check whether the flashloader is valid and start the application instead if not.

If you receive the new image in the flashloader, whatever functionality your application normally provides is not available for as long as you are in the flashloader. If your device is somewhere remote and only has a low bandwidth connection (maybe LoRa or similar), transferring the new image could take a significant amount of time. Being able to receive it whilst maintaining normal functionality is probably a plus. If you have a fast connection or can live with the device doing nothing other than wait for data to arrive, it doesn't matter.

What I'm trying to say is that there is no one-size-fits-all solution for any of this. What is not a problem in one project could be an absolute deal-breaker in another. There will be projects that are not suitable, but I believe the functionality currently provided by the flashloader (i.e. 'flashloader.c') is good enough to be usable by a lot of projects. Every one of those projects will have to decide how a new image can be transferred to the device and how best to write it to flash ready for the flashloader to do its job.

'app.c' is an example to be looked at and replaced. I don't consider it part of the "flashloader".