AUR samtools installing problem

rnz-v001 · 2015-11-21 21:10:40

Hello folk,
I just tried several time to install samtools, I tried with yaourt, packer and manaully. I got the same error message:

Samtools mpileup tests:

EXPECTED FAIL: Task failed, but expected to fail;
when running $samtools mpileup -x -d 8500 -B -f mpileup.ref.fa deep.sam|awk '{print $4}'
Error running $samtools mpileup -u -x -f mpileup.ref.fa mpileup.1.bam | GREP_OPTIONS= egrep -v '^##samtools'

UNEXPECTED FAIL: Output mismatch for $samtools mpileup -x -F 0.60 -u -f mpileup.ref.fa indels.bam|$filter|awk '/INDEL/'
See FAIL-59.out.1 vs expected/59.out

UNEXPECTED FAIL: Output mismatch for $samtools mpileup -x -F 0.60 -u -f mpileup.ref.fa indels.cram|$filter|awk '/INDEL/'
See FAIL-59.out.2 vs expected/59.out

Expected   passes:   120
Unexpected passes:   0
Expected   failures: 1
Unexpected failures: 2
=> FAIL
Makefile:184: set di istruzioni per l'obiettivo "check" non riuscito
make: *** [check] Errore 1
==> ERRORE: Si è verificato un errore in check().
    L'operazione sta per essere interrotta...
The build failed.

I read the comment on AUR but that didn't work with me. I don't know If it's a problem from the package or my machine...

any ideas?

thanks

Trilby · 2015-11-21 21:18:39

rnz-v001 wrote:

I don't know If it's a problem from the package or my machine...

I have no issues with samtools. I just rebuilt with makepkg and all went fine.

What's in the files it tells you to compare?

rnz-v001 · 2015-11-21 23:03:07

those two FAIL-59.out.* are both empty expected/59.out it's just this:

17	547	.	A	AGCATCCTGCG	0	.	INDEL;I16=2,0,1,2,404,81608,538,130834,120,7200,180,10800,38,722,63,1347;QS=1.01515,0.984848	PL	111,0,51	22,0,54

I don't think those files matter

Trilby · 2015-11-21 23:32:55

What do you mean they don't matter? Of course they matter. The checks run to ensure the tools work properly.

The 'expected' shows the read that should be produced when samtools is run, but the empty file indicates that it is not being produced: samtoolsl is not doing what it is supposed to, so the check fails and the faulty package is not installed.

rnz-v001 · 2015-11-21 23:46:08

mm yeah you're right, but how to undestand the problem reading that? :\

Trilby · 2015-11-22 00:09:21

This means the following command is failing to show the indels that it should:

samtools mpileup -x -F 0.60    -u -f mpileup.ref.fa indels.bam | ../vcf-miniview -f

I'm not particularly well versed in samtools, but if I had this error, I'd break down that pipeline: check the output of samtools itself, then check whether vcf-miniview is working.

rnz-v001 · 2015-11-22 11:32:25

so executing the following

./samtools mpileup -x -F 0.60 -u -f test/mpileup/mpileup.ref.fa test/mpileup/indels.bam >> output_test.log

I got this: http://pastebin.com/Huh2fTt8

instead by the following

./samtools mpileup -x -F 0.60 -u -f test/mpileup/mpileup.ref.fa test/mpileup/indels.bam | ./test/vcf-miniview  -f >> output_test2.log

this: http://pastebin.com/8KJWdKRh

as far as I know it seems to work... :\

Trilby · 2015-11-22 13:40:55

The commands executed completely - but it didn't "work". That is about the worst outcome. If it just errored out it would be easier to pinpoint why it wasn't running. But it is running and it is giving you inaccurate results. I get roughly 150 variant call differences between your output and mine - the most notable for this test is that your output does not include any indels, and there is one at position 547. The miss of the indel is likely what leads to the other 150 or so call errors (edit: just confirmed, the other errors are in everything downstream of the indel).

I don't think you want a samtool package that will give you incorrect interpretations from very expensive data.

I'm trying to test vcf-miniview on your bcf versus mine to track this further, but it seems the file sharing site you posted it on butchered the binary format. If you can email the raw bcf (output from the first step you did above) to the address linked in my profile (to the left, or just below) I'll see if I can dig a little deeper.

Also, could you provide a bit of information on your hardware and running environment?

rnz-v001 · 2015-11-22 14:37:15

I see, really awkward...

I just sent you an e-mail

here some hw info:

Architecture:          i686
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                2
On-line CPU(s) list:   0,1
Thread(s) per core:    1
Core(s) per socket:    2
Socket(s):             1
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 23
Model name:            Intel(R) Core(TM)2 Duo CPU     T6400  @ 2.00GHz
Stepping:              10
CPU MHz:               1600.000
CPU max MHz:           2000.0000
CPU min MHz:           1200.0000
BogoMIPS:              3990.71
Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm constant_tsc arch_perfmon pebs bts aperfmperf pni dtes64 monitor ds_cpl est tm2 ssse3 cx16 xtpr pdcm sse4_1 xsave lahf_lm dtherm

Linux firefly 4.1.13-1-lts #1 SMP Tue Nov 10 10:07:54 CET 2015 i686 GNU/Linux

Trilby · 2015-11-22 15:04:53

Thanks. Unfortunately that confirmed one suspicion and lended further likelihood to another.

First, vcf-miniview is behaving fine - which is unfortunate as that tool does not actually get installed. If that were the culprit you might be able to move forward anyway. Unfortunately the bcf that samtools is producing on your machine is infact incorrect.

Further I asked about hardware information to see if anything jumped out - but my biggest suspicion was about processor architecture. I wouldn't be surprised if this was an i686-specific bug, and the fact that you are running i686 is - at least - consistent with that suspicion.

~~I think it's time to collect some of this information and send it on to the samtools mailing list, and potentially a bug report.~~

As a workaround - if you're up for it - your processor is perfectly capable of running x86_64. I am currently on a Core 2 Duo myself (P8600 here). So if you migrated to a x86_64 install, I suspect you could avoid this issue.

And following from that - I tinker with some of these tools on my core 2 duo machine, but I don't think I'd ever be patient enough to do any real analyses on it. Do you intend to use samtools on that machine, or are you just trying it out?

EDIT: scratch the part about the mailing list - it seems these issues are already heavily discussed on the github issue tracker. Perhaps the AUR package should not list i686 as a supported architecture until these issues are resolved (if they every are).

(one relevant example)

EDIT2: this was fixed! But it is in line for the 1.3 release. The AUR still has 1.2 where the problem existed.

rnz-v001 · 2015-11-22 15:44:18

uh! thanks a lot.

Yes, I need to work with that software on my machine. So what you suggest? to move to an x86_64 install or just wait a new release?

also the official site still have the 1.2 http://www.htslib.org/download/

Trilby · 2015-11-22 16:03:54

I suspect you'll find i686 bugs - especially in computationally intensive software - could be very slow in being fixed. If I were you I'd switch to x86_64. But note you will need to backup *everything* as you'd be reinstalling arch from scratch. You could maintain your home parition if that is separate - but you'd still want a backup just in case.

I have no idea when 1.3 might be released.

As a third option, you could build from git, but this would be only for in-house analyses. If you intend to publish anything, I'd definitely not use developmental code.

Arch Linux

#1 2015-11-21 21:10:40

AUR samtools installing problem

#2 2015-11-21 21:18:39

Re: AUR samtools installing problem

#3 2015-11-21 23:03:07

Re: AUR samtools installing problem

#4 2015-11-21 23:32:55

Re: AUR samtools installing problem

#5 2015-11-21 23:46:08

Re: AUR samtools installing problem

#6 2015-11-22 00:09:21

Re: AUR samtools installing problem

#7 2015-11-22 11:32:25

Re: AUR samtools installing problem

#8 2015-11-22 13:40:55

Re: AUR samtools installing problem

#9 2015-11-22 14:37:15

Re: AUR samtools installing problem

#10 2015-11-22 15:04:53

Re: AUR samtools installing problem

#11 2015-11-22 15:44:18

Re: AUR samtools installing problem

#12 2015-11-22 16:03:54

Re: AUR samtools installing problem

Board footer