Browsed by
Category: Science

Creating weighted tables with R / sum of numerics associated to some categorical variable

Creating weighted tables with R / sum of numerics associated to some categorical variable

The normal table command table() calculates the frequency of each element of a vector like this:

R> df <- data.frame(var = c("A", "A", "B", "B", "C", "C", "C"))
R> table(df)
A B C 
2 2 3 

So, it tells us, we have two times A and B and three times C, accordingly.

However, if we have now the situation like this:

df <- data.frame(var = c("A", "A", "B", "B", "C", "C", "C"), value = c(10, 20, 20, 40, 15, 25, 35))

Meaning, we have a categorical variable var and a numeric variable value and for each categorical variable we would like to get the sum over the numerical variable, we can simply use the base-R command aggregate like this

R> aggregate(value ~ var, data = df, FUN = sum)
  var wt
1   A 40
2   B 60
3   C 70

As I often use also the data.table package, here is also a simple solution using this package, assuming we do (respective have a data table from some other source, like fread)

dt <- data.table(df)

Then we can just sume over a column name with respect to another column like this (and assign the value into a new variable tot) :

setDT(dt)[, .(n = sum(value)), var] 
Couldn’t delete a failed R package installation

Couldn’t delete a failed R package installation

Today I faced a strange thing, while I tried to install an update to one of my R-packages. As usually, I installed the latest version from it like this


But the installation failed, and when I tried to reinstall it, I got this error message:

Installing package into ‘/homeappl/home/<user>/R/x86_64-redhat-linux-gnu-library/4.1’
(as ‘lib’ is unspecified)
ERROR: failed to lock directory ‘/homeappl/home/<user>/R/x86_64-redhat-linux-gnu-library/4.1’ for modifying
Try removing ‘/homeappl/home/<user>/R/x86_64-redhat-linux-gnu-library/4.1/00LOCK-GenomicTools’
Warning message:
In i.p(...) :
  installation of package ‘/tmp/Rtmp<...>Lv/file3c36<...>34/GenomicTools_0.2.11.tar.gz’ had non-zero exit status

So, I tried to go in said directory and delete the folder manually and there I received another error:

rm: cannot remove '00LOCK-GenomicTools/GenomicTools/libs/.nfs00000001002e2<...>d': Device or resource busy

I tried this and this, but nothing helped to delete that folder, it kept mocking my that the device is busy. Eventually, it helped just to rename the folder list this

mv 00LOCK-GenomicTools/ 00LOCK-GenomicTools-deleteThis/

It is now still hanging there in the folder, but I was able to reinstall the R-package and now I need to revisit the folder in a few days and check, if the device is still busy or if I can delete it then…

Workshop: “Foundation for the Future Agenda”

Workshop: “Foundation for the Future Agenda”

I just noticed I haven’t written anything yet about my attendance at the Shared Workshop: “Foundation for the Future Agenda” in Hinxton, UK, during February. This was a workshop organized by EBI for the three H2020 projects GeneSwitch, BovReg and AquaFAANG.

As always, it was an excellent meeting in Hinxton, with lots of good interaction and talks. Also, a visit to the Red Lion is always a nice experience.

Take a random sample of size k from paired-end FASTQ

Take a random sample of size k from paired-end FASTQ

Today I wrote a bash script that creates a random subset of a paired-end FASTQ file pair. It requires the names of the two FASTQ-files as input and also the amount of reads that the sample should have.

The script is mainly based on this Blog post. This is a rather rough code and it could be more user-friendly and allow for more options, but in its current form, it does what I need it to do.


round() {
    printf "%.2f" "$1"

# Input test
 if ! [[ $sample =~ ^-?[0-9]+([.][0-9]+)?$ ]]; then 
>&2 echo "$sample is not a number"; exit 1; 


if [ $extension1 == "gz" ]; then
  gunzip $file1;
if [ $extension2 == "gz" ]; then
  gunzip $file2;

lines=$(wc -l < $file1)
echo $lines
echo $sample

if (( $(awk 'BEGIN {print ("'$sample'" <= 1)}') )); then
  sample=$(awk 'BEGIN {printf("%.0f", "'$sample'" * "'$lines'")}')

echo $sample

paste $file1 $file1 | \
awk '{ printf("%s",$0); n++; if(n%4==0) { printf("\n");} else { printf("\t");} }' | \
awk -v k=$sample 'BEGIN{srand(systime() + PROCINFO["pid"]); }{ s=x++<k?x- 1:int(rand()*x);
                  if(s<k)R[s]=$0}END{for(i in R)print R[i]}' | \
awk -F"\t" -v file1=$fn1 -v file2=$fn2 '{print $1"\n"$3"\n"$5"\n"$7 > file1;\
                                         print $2"\n"$4"\n"$6"\n"$8 > file2}'
if [ $extension1 == "gz" ]; then
  gzip $fn1;
  gzip $file1;
if [ $extension2 == "gz" ]; then
  gzip $fn2;
  gzip $file2;