[solved]inserting text into a file using Perl.

Cyrusm · 2010-10-20 12:55:57

I need some help with a perl script that I've been hacking together. basically what the script does now is it reads in a string from an input file, runs it through a hash, and outputs a large string of hex values and some other useful data. what I'm doing is taking this string of comma delimited hex values and copy and pasting them into an array in a piece of C code that I have for my microcontroller that displays a scrolling marquis on an LED matrix. What I would like to do essentially is to automate this copy/paste process and just have the output of the perl script shoved directly between the brackets. (at which point I can tie the whole build process together with bash script and call it a day.)

this is my first ever attempt at writing a perl script, so it's not pretty

#!/usr/bin/env perl
use strict;
use warnings;

# my huge hash of hex values for each letter

my %hashkey = (
    " " => "0x00,0x00,0x00,0x00,",
    "A" => "0x1F,0x28,0x48,0x28,0x1F,0x00,0x00,",
    "B" => "0x7F,0x49,0x49,0x36,0x00,0x00,",
    "C" => "0x3E,0x41,0x41,0x41,0x22,0x00,0x00,",
    "D" => "0x7F,0x81,0x81,0x3E,0x00,0x00,",
    "E" => "0x7F,0x49,0x49,0x49,0x00,0x00,",
    "F" => "0x7F,ox48,0x48,0x40,0x00,0x00,",
    "G" => "0x3E,0x41,0x41,0x45,0x26,0x00,0x00,",
    "H" => "0x7F,0x08,0x08,0x08,0x7F,0x00,0x00,",
    "I" => "0x41,0x41,0x7F,0x41,0x41,0x00,0x00,",
    "J" => "0x02,0x01,0x01,0x3E,0x00,0x00,",
    "K" => "0x7F,0x08,0x14,0x22,0x41,0x00,0x00,",
    "L" => "0x7F,0x01,0x01,0x01,0x00,0x00,",
    "M" => "0x7F,0x20,0x10,0x20,0x7F,0x00,0x00,",
    "N" => "0x7F,0x20,0x08,0x02,0x7F,0x00,0x00,",
    "O" => "0x1C,0x22,0x41,0x41,0x22,0x1C,0x00,0x00,",
    "P" => "0x7F,0x48,0x48,0x60,0x00,0x00,",
    "Q" => "0x1C,0x22,0x41,0x49,0x22,0x1D,0x00,0x00,",
    "R" => "0x7F,0x48,0x4C,0x33,0x00,0x00,",
    "S" => "0x32,0x49,0x49,0x26,0x00,0x00,",
    "T" => "0x40,0x40,0x7F,0x40,0x40,0x00,0x00,",
    "U" => "0x7E,0x01,0x01,0x01,0x7E,0x00,0x00,",
    "V" => "0x7C,0x02,0x01,0x02,0x7C,0x00,0x00,",
    "W" => "0x7F,0x02,0x04,0x02,0x7F,0x00,0x00,",
    "X" => "0x63,0x14,0x08,0x14,0x63,0x00,0x00,",
    "Y" => "0x40,0x20,0x1F,0x20,0x40,0x00,0x00,",
    "Z" => "0x43,0x45,0x49,0x51,0x61,0x00,0x00,",
    "." => "0x02,0x00,0x00,",
    "!" => "0x7C,0x00,0x00,",
    "?" => "0x5A,0x50,0x70,0x00,0x00,",
    "0" => "0x3E,0x43,0x49,0x61,0x3E,0x00,0x00,",
    "1" => "0x21,ox7F,0x01,0x00,0x00,",
    "2" => "0x21,0x43,0x45,0x49,0x31,0x00,0x00,",
    "3" => "0x22,0x14,0x94,0x55,0x22,0x00,0x00,",
    "4" => "0x18,0x28,0x48,0x7F,0x08,0x00,0x00,",
    "5" => "0x79,0x49,0x49,0x4F,0x00,0x00,",
    "6" => "ox3E,0x49,0x,49,0x26,0x00,0x00,",
    "7" => "0x40,0x47,0x58,0x60,0x00,0x00,",
    "8" => "0x36,0x49,0x49,0x36,0x00,0x00,",
    "9" => "0x30,0x48,0x48,0x3F,0x00,0x00,",
    "\n"=> "",
);

# Open up my input and output files
open(my $in, "<", "string.txt") or die "Can't open string.txt: $!";
open(my $out, ">","output.txt") or die "something went sour: $!";

# Read in the string from the input file
my $string = <$in>;

# convert string to upper case
$string = uc($string);

# convert string to array of characters
my @line = split('',$string);

#add a leading space
print $out $hashkey{" "};

# take each character and print it's hex matrix to the outfile
foreach (@line)
{
    print $out $hashkey{$_};
}

# add a trailing space
print $out $hashkey{" "};

# close all files
close($out);
close($in);

# reopen output file as input and make a string out of it.
open(my $in2, "<","output.txt") or die "failed opening output.txt: $!";
my $hexstring = <$in2>;

# split the string into an array at the commas and count number of array elements.
my @countwords = split(',',$hexstring);
my $count = @countwords;
$count -= 1;
close($in2);
open(my $out2, ">>", "output.txt") or die "failed to write to output.txt: $!";
print $out2 "number of elements = ",$count;
close($out2);

And here's the location in the C file I would like to output to:

//=====================
//HEADERS
//=====================
#include <avr/io.h>
#include <avr/interrupt.h>

//======================
//CONSTANTS
//======================

#define MAX 77 //length of Marquis, manually calculated.



//======================
void ioinit(void);          //Initializes IO
void timer_0_set_up(void);     //Initialize Timer 0
void timer_2_set_up(void);     //Initialize Timer 2
//======================

/* I WANT TO STICK THE PERL OUTPUT INTO THIS ARRAY! */
volatile const uint8_t marquis[] = {0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* space */
                    0x7F,0x08,0x08,0x08,0x7F,0x00,0x00,      /* H */
                    0x7F,0x49,0x49,0x49,0x00,0x00,         /* E */
                    0x7F,0x01,0x01,0x01,0x00,0x00,         /* L */
                    0x7F,0x01,0x01,0x01,0x00,0x00,           /* L */
                    0x1C,0x22,0x41,0x41,0x22,0x1C,0x00,0x00, /* O */
                    0x00,0x00,0x00,0x00,             /* _ */
                    0x7F,0x02,0x04,0x02,0x7F,0x00,0x00,      /* W */
                    0x1C,0x22,0x41,0x41,0x22,0x1C,0x00,0x00, /* O */
                    0x7F,0x48,0x4C,0x33,0x00,0x00,         /* R */
                    0x7F,0x01,0x01,0x01,0x00,0x00,           /* L */
                    0x7F,0x81,0x81,0x3E,0x00,0x00,         /* D */
                    0x7C};                     /* ! */

volatile uint8_t position = 0;


int main (void)
{    
    ioinit();
    timer_0_set_up();
    timer_2_set_up();

    sei();

    while(1)
    {
        //wait for interrupt.    
    }

    return(0);
}

void ioinit (void)
{
    //1 = output, 0 = input
    DDRB = 0b11111111; //All outputs
    DDRC = 0b11111111; //All outputs
    DDRD = 0b11111111; //PORTD (RX on PD0)
}

/* Timer 0 output compare interrupt every 18 Hz to scroll Marquis */
void timer_0_set_up(void)
{
    TIMSK0 = _BV(OCF0A); //enable output compare interrupt
    TCCR0A = _BV(WGM01); //enable CTC mode
    TCCR0B = _BV(CS02);  //set prescaler to clk/256
    OCR0A  = 217;         //increment every 18Hz.
}
/* Timer 2 output compare interrupt every 10Hz to multiplex the display */
void timer_2_set_up(void)
{
    TIMSK2 = _BV(OCF2A); //enable output compare interrupt
    TCCR2A = _BV(WGM01); //enable CTC mode
    TCCR2B = _BV(CS02) | _BV(CS01); //set prescaler to clk/1024
    OCR2A  = 98;         //increment every 10Hz
}

ISR(TIMER0_COMPA_vect)
{
    /* increment the position to scroll the text */

    if ((position + 7) == MAX)//prevent array overflow, restart marquis
    {
        position = 0;
    }
    position++;
}
ISR(TIMER2_COMPA_vect)
{

    /* multiplex the LED Matrix and display the value
     * stored to PortB on the given column*/

    for (uint8_t offset = 0; offset < 8; offset++)
    {
        PORTD = offset;
        PORTB = marquis[position + offset];
    }
}

Last edited by Cyrusm (2010-10-20 20:28:59)

tlvb · 2010-10-20 16:18:49

I did do something similar a while ago, basically by creating markers in the file for the part of text to be replaced as in
this example

/* foo.c */
/* ... */
/* BEGIN PERLGEN DATA */
char* c = "old stuff";
/* END PERLGEN DATA */
/* ... */

# bar.pl
use strict;
use warnings;
use Tie::File;

my @values;

tie my @fh, 'Tie::File', "foo.c" or die $!;

@values = "char* c = \"new stuff\";"

my $first = (grep {$fh[$_] =~ /\/\* BEGIN PERLGEN DATA \*\//} 0..$#fh)[0]+1;
my $last = (grep {$fh[$_] =~ /\/\* END PERLGEN DATA \*\//} 0..$#fh)[0];

splice @fh, $first, $last-$first, @values;
untie @fh;

The example above will most likely not work at all, and may be syntactically incorrect in many places, but it might
give you some insight in what functions to use. The original code is below, but is a little finicky because I thought
it was a neat idea to write the perl and the c code it modifies in the same file, so depending on if you run
the code with the perl interpreter or through gcc it will either update the c part or compile the c part.

You may ignore the c part, and skip down to the perl at the bottom, though make note of the /* BEGIN/END PERLFONT */
and how it is grepped for later in the perl code, also the 'my @lines...' and the for loop in the perl code is only there to
generate the font data from the #> comments in the perl code.

#if 0
=pod
#endif

#include <avr/io.h>
#define F_CPU 1000000
#include <util/delay.h>
#include <avr/pgmspace.h>
#include <avr/interrupt.h>
#define IN PD0
#define CK PD1
#define RS PD2

/* BEGIN PERLFONT */
const uint8_t font[] PROGMEM = {
0x00,0x7c,0x82,0x92,0x82,0x7c,0x00,0x84,0x84,0xfe,0x80,0x80,
0x00,0x84,0xc2,0xa2,0x92,0x8c,0x00,0x44,0x82,0x92,0x92,0x6c,
0x00,0x1e,0x10,0x10,0x10,0xfe,0x00,0x5e,0x92,0x92,0x92,0x62,
0x00,0x7c,0x92,0x92,0x92,0x60,0x00,0x02,0x02,0xe2,0x12,0x0e,
0x00,0x6c,0x92,0x92,0x92,0x6c,0x00,0x0c,0x92,0x92,0x92,0x7c
};
/* END PERLFONT */

uint8_t dispctrtable[8] = {3,4,2,1,7,6,0,5};
uint8_t colxtable[12] = {0,0,0,1,1,1,2,2,2,3,3,3};

volatile uint8_t dispctr = 0;
volatile uint8_t fb0[4] = {2,3,4,0};
volatile uint8_t fb1[4] = {2,3,4,0};
volatile uint8_t fbl[4] = {0,0,0,0};

int main(void) {

    TCCR0A = _BV(WGM00);
    TCCR0B = _BV(CS00);
    TIMSK = _BV(TOIE0);

    DDRD = _BV(PD0)|_BV(PD1)|_BV(PB2);
    DDRB = 0xff;

    PORTD = _BV(RS)|_BV(IN);

    for (uint8_t i=0; i<8; i++) {
        PORTD |=  _BV(CK);
        PORTD &=~ _BV(CK);
    }

    sei();

    uint8_t timer = 0;

    for (;;) {
        if (timer==19) {
            timer = 0;
            if(fb1[3]<9)
                fb1[3]++;
            else
                fb1[3] = 0;
        }
        else {
            timer++;
        }
        for (uint8_t i=0; i<4; i++) {
            if (fb0[i] != fb1[i]) {
                if (fbl[i] < 11) {

                    if (fb1[i] == 0 && i>0 && fbl[i] == 1) {
                        if (
                            (i==3 && fb0[i-1]<5) ||
                            (i==2 && fb0[i-1]<9 && fb0[i-2]<2) ||
                            (i==2 && fb0[i-1]<3) ||
                            (i==1 && fb0[i-1]<2)
                            )
                            fb1[i-1] = fb0[i-1]+1;
                        else
                            fb1[i-1] = 0;
                    }

                    fbl[i]++;
                }
                else {
                    fb0[i] = fb1[i];
                    fbl[i] = 0;
                }
            }
        }
        _delay_ms(25);
    }
}

ISR(TIMER0_OVF_vect) {
    PORTB = 0;
    if (dispctr == 23) {
        dispctr = 0;
        PORTD &=~ _BV(IN);
    }
    else {
        dispctr++;
        PORTD |= _BV(IN);
    }
    uint8_t realx, fbc, fbco, byte0, byte1;
    realx = dispctrtable[dispctr&7]|(dispctr&~7);
    fbc = colxtable[realx>>1];
    fbco = realx-fbc*6;

    PORTD |=  _BV(CK);
    PORTD &=~ _BV(CK);

    byte0 = pgm_read_byte(&font[fb0[fbc]*6+fbco]);
    byte1 = pgm_read_byte(&font[fb1[fbc]*6+fbco]);

    //PORTB = pgm_read_byte(&font[dispctrtable[col&7]|(col&~7)]);
    PORTB = (byte0<<fbl[fbc])|(byte1>>(11-fbl[fbc]));
}

#if 0

=cut

#>............................................................
#>..000....0....000...000..0...0.00000..000..00000..000...000.
#>.0...0.000...0...0.0...0.0...0.0.....0.........0.0...0.0...0
#>.0...0...0.......0.....0.0...0.0.....0.........0.0...0.0...0
#>.0.0.0...0......0....00..00000.0000..0000.....0...000...0000
#>.0...0...0.....0.......0.....0.....0.0...0...0...0...0.....0
#>.0...0...0....0....0...0.....0.0...0.0...0...0...0...0.....0
#>..000..00000.00000..000......0..000...000....0....000...000.

use strict;
use warnings;
use Tie::File;

my @values;

tie my @fh, 'Tie::File', "$0" or die $!;


my @lines = grep /^#>/, @fh;
while (@lines>8) {
    $lines[$_] .= substr(pop @lines, 2) for (reverse 0..7);
} 

for my $i (0..length($lines[0])-3) {
    my $val=0;
    for (0..7) {
        $val += 1<<$_ if substr($lines[$_],$i+2,1) eq '0';
    }
    push @values, sprintf('0x%02x',$val);
}

@values = join(',', @values);
unshift @values, "const uint8_t font[] PROGMEM = {";
push @values, "};";
my $first = (grep {$fh[$_] =~ /\/\* BEGIN PERLFONT \*\//} 0..$#fh)[0]+1;
my $last = (grep {$fh[$_] =~ /\/\* END PERLFONT \*\//} 0..$#fh)[0];

splice @fh, $first, $last-$first, @values;
untie @fh;

#endif

The full description of the perl part of the program is that it takes the file (itself) searches for the lines that begins with "#>" which contains the graphical data
that should be turned into hex values describing each column of pixels (eg, in the above code the first column "........" will be 0x00,
the second column "..00000." will be 0x7C etc etc) then it searches for the BEGIN/END PERLFONT c comments and replaces the text between it with the
generated hex data suitably formatted as a progmem variable declaration. (It seems though that I have mixed up the msb and the lsb of the data
compared to what I intended to do, but then coded around it in the c code...)

Last edited by tlvb (2010-10-20 16:32:29)

Cyrusm · 2010-10-20 17:18:40

Hey, thanks a lot! it's not exactly what I was looking for but pretty damn close I'd never considered the possibility of having the perl and C in the same file and I may look into playing with that eventually. but for the time being I'll try to keep my languages separate. This should give me a good push in the right direction!

I'll mark this thread as solved once I come up with a working solution, I'm still open to ideas if anybody has them

Daenyth · 2010-10-20 17:35:15

Assuming your script output is directly able to be copy pasted into your editor... If you're using vim this is as easy as ":r! perl myscript.pl". Do that on the line above where you want it to be inserted.

tlvb · 2010-10-20 19:15:22

A more concrete example of what I wished to show you before:

use strict;
use warnings;
use Tie::File;

my $codefile = 'code.c';
my $arraystr = 'volatile const uint8_t marquis[] = {';
my @datalines = ( 
        'These are the datalines that',
        'you generate from your text',
        'file or any other way you',
        'see fit.');

tie my @code, 'Tie::File', $codefile or die "Eep! $!";

my $start = (grep {$code[$_] eq $arraystr} 1..$#code)[0]+1;
my $length = (grep {$code[$_] eq '};' and $_>$start} 1..$#code)[0] - $start;

splice @code, $start, $length, @datalines;

untie @code;

/* code.c */
This will remain.
volatile const uint8_t marquis[] = { 
    This is the stuff that will be
    replaced.
};
This will remain.

-->

/* code.c */
This will remain.
volatile const uint8_t marquis[] = { 
These are the datalines that
you generate from your text
file or any other way you 
see fit.
};
This will remain.

Last edited by tlvb (2010-10-20 19:31:18)

Cyrusm · 2010-10-20 20:28:27

wow, you did all the hard work for me, thanks for saving me some time/effort! works exactly as I was hoping it would.

on an aside, I've never played with Perl until yesterday, and I'm going to have to say that at least for this purpose (quick and dirty text modifications) I've never come across a more flexible, powerful, well documented,and easy to learn language. very exciting. it looks like I've got a new toy to play with

Arch Linux

#1 2010-10-20 12:55:57

[solved]inserting text into a file using Perl.

#2 2010-10-20 16:18:49

Re: [solved]inserting text into a file using Perl.

#3 2010-10-20 17:18:40

Re: [solved]inserting text into a file using Perl.

#4 2010-10-20 17:35:15

Re: [solved]inserting text into a file using Perl.

#5 2010-10-20 19:15:22

Re: [solved]inserting text into a file using Perl.

#6 2010-10-20 20:28:27

Re: [solved]inserting text into a file using Perl.

Board footer