Partially deallocate malloc'd buffer in C?

Basu · 2011-09-23 03:42:53

I have a question about freeing malloc'd buffers. If I use malloc to allocated a section of memory is there some way I can free a specific portion of it later? I tried passing an offset pointer to free() but got an error:

a.out(14434) malloc: *** error for object 0x10bb0095e: pointer being freed was not allocated

My use case is that I want to malloc a buffer to be filled by read() from a file descriptor but then truncate the allocated memory based on the number of bytes actually read. Is there some general idiom (or function) for doing that?
Thanks,
Basu

djgera · 2011-09-23 03:52:19

Use realloc(3) , see manpage for details.

tavianator · 2011-09-23 04:06:02

The general idiom is to use fseek() followed by ftell(), or stat(), to determine the file's size.

The general idiom is also to not blindly read an entire file into memory, though, 'cause files can be huge. mmap() will accomplish the same thing but without forcing a read until it's actually needed. And working with the file as a stream (read some data, do something, discard, rinse, repeat) is best if possible.

Basu · 2011-09-23 04:16:26

I'm reading from a TCP socket. I want to grab the data from a read and stash it in a queue. I don't want to waste space if I can free memory once I know how big the read actually was. The other option I thought of was having a large, fixed size buffer that's allocated once for all reads and then doing a precise memcpy (using the return from read) for each push to the queue. That might be a cleaner solution than a heap malloc and realloc for each read.

tavianator · 2011-09-23 05:39:36

Wow sorry, I read fread() rather than read() so didn't even think about sockets. Well what I'd maybe use is a dynamic array, which starts small and doubles in size every time it hits capacity (here's one I wrote). Re-use the space for each read.

Ideally you'd just get the size as the first thing pushed over the wire and then allocate your buffer, but I'm guessing that's not possible?

Last edited by tavianator (2011-09-23 05:40:07)

Arch Linux

#1 2011-09-23 03:42:53

Partially deallocate malloc'd buffer in C?

#2 2011-09-23 03:52:19

Re: Partially deallocate malloc'd buffer in C?

#3 2011-09-23 04:06:02

Re: Partially deallocate malloc'd buffer in C?

#4 2011-09-23 04:16:26

Re: Partially deallocate malloc'd buffer in C?

#5 2011-09-23 05:39:36

Re: Partially deallocate malloc'd buffer in C?

Board footer