EMBOSS: fdnacomp

fdnacomp

Wiki

The master copies of EMBOSS documentation are available at http://emboss.open-bio.org/wiki/Appdocs on the EMBOSS Wiki.

Please help by correcting and extending the Wiki pages.

Function

DNA compatibility algorithm

Description

Estimates phylogenies from nucleic acid sequence data using the compatibility criterion, which searches for the largest number of sites which could have all states (nucleotides) uniquely evolved on the same tree. Compatibility is particularly appropriate when sites vary greatly in their rates of evolution, but we do not know in advance which are the less reliable ones.

Algorithm

This program implements the compatibility method for DNA sequence data. For a four-state character without a character-state tree, as in DNA sequences, the usual clique theorems cannot be applied. The approach taken in this program is to directly evaluate each tree topology by counting how many substitutions are needed in each site, comparing this to the minimum number that might be needed (one less than the number of bases observed at that site), and then evaluating the number of sites which achieve the minimum number. This is the evaluation of the tree (the number of compatible sites), and the topology is chosen so as to maximize that number.

Compatibility methods originated with Le Quesne's (1969) suggestion that one ought to look for trees supported by the largest number of perfectly fitting (compatible) characters. Fitch (1975) showed by counterexample that one could not use the pairwise compatibility methods used in CLIQUE to discover the largest clique of jointly compatible characters.

The assumptions of this method are similar to those of CLIQUE. In a paper in the Biological Journal of the Linnean Society (1981b) I discuss this matter extensively. In effect, the assumptions are that:

Each character evolves independently.
Different lineages evolve independently.
The ancestral base at each site is unknown.
The rates of change in most sites over the time spans involved in the the divergence of the group are very small.
A few of the sites have very high rates of change.
We do not know in advance which are the high and which the low rate sites.

That these are the assumptions of compatibility methods has been documented in a series of papers of mine: (1973a, 1978b, 1979, 1981b, 1983b, 1988b). For an opposing view arguing that arguments such as mine are invalid and that parsimony (and perhaps compatibility) methods make no substantive assumptions such as these, see the papers by Farris (1983) and Sober (1983a, 1983b, 1988), but also read the exchange between Felsenstein and Sober (1986).

There is, however, some reason to believe that the present criterion is not the proper way to correct for the presence of some sites with high rates of change in nucleotide sequence data. It can be argued that sites showing more than two nucleotide states, even if those are compatible with the other sites, are also candidates for sites with high rates of change. It might then be more proper to use DNAPARS with the Threshold option with a threshold value of 2.

Change from an occupied site to a gap is counted as one change. Reversion from a gap to an occupied site is allowed and is also counted as one change. Note that this in effect assumes that a gap N bases long is N separate events. This may be an overcorrection. When we have nonoverlapping gaps, we could instead code a gap as a single event by changing all but the first "-" in the gap into "?" characters. In this way only the first base of the gap causes the program to infer a change.

If the U (User Tree) option is used and more than one tree is supplied, the program also performs a statistical test of each of these trees against the one with highest likelihood. If there are two user trees, the test done is one which is due to Kishino and Hasegawa (1989), a version of a test originally introduced by Templeton (1983). In this implementation it uses the mean and variance of weighted compatibility differences between trees, taken across sites. If the two trees compatibilities are more than 1.96 standard deviations different then the trees are declared significantly different.

If there are more than two trees, the test done is an extension of the KHT test, due to Shimodaira and Hasegawa (1999). They pointed out that a correction for the number of trees was necessary, and they introduced a resampling method to make this correction. In the version used here the variances and covariances of the sum of weighted compatibilities of sites are computed for all pairs of trees. To test whether the difference between each tree and the best one is larger than could have been expected if they all had the same expected compatibility, compatibilities for all trees are sampled with these covariances and equal means (Shimodaira and Hasegawa's "least favorable hypothesis"), and a P value is computed from the fraction of times the difference between the tree's value and the highest compatibility exceeds that actually observed. Note that this sampling needs random numbers, and so the program will prompt the user for a random number seed if one has not already been supplied. With the two-tree KHT test no random numbers are used.

In either the KHT or the SH test the program prints out a table of the compatibility of each tree, the differences of each from the highest one, the variance of that quantity as determined by the compatibility differences at individual sites, and a conclusion as to whether that tree is or is not significantly worse than the best one.

The algorithm is a straightforward modification of DNAPARS, but with some extra machinery added to calculate, as each species is added, how many base changes are the minimum which could be required at that site. The program runs fairly quickly.

Usage

Here is a sample session with fdnacomp

% fdnacomp -ancseq -stepbox -printdata DNA compatibility algorithm Input (aligned) nucleotide sequence set(s): dnacomp.dat Phylip tree file (optional): Phylip weights file (optional): Phylip dnacomp program output file [dnacomp.fdnacomp]: Adding species: 1. Alpha 2. Beta 3. Gamma 4. Delta 5. Epsilon Doing global rearrangements !---------! ......... Output written to file "dnacomp.fdnacomp" Trees also written onto file "dnacomp.treefile"

Go to the input files for this example
Go to the output files for this example

Example 2

% fdnacomp DNA compatibility algorithm Input (aligned) nucleotide sequence set(s): dnacomp.dat Phylip tree file (optional): dnacomptree.dat Phylip weights file (optional): Phylip dnacomp program output file [dnacomp.fdnacomp]: Output written to file "dnacomp.fdnacomp" Trees also written onto file "dnacomp.treefile"

Go to the input files for this example
Go to the output files for this example

Command line arguments

DNA compatibility algorithm
Version: EMBOSS:6.5.0.0

   Standard (Mandatory) qualifiers:
  [-sequence]          seqsetall  File containing one or more sequence
                                  alignments
  [-intreefile]        tree       Phylip tree file (optional)
   -weights            properties Phylip weights file (optional)
  [-outfile]           outfile    [*.fdnacomp] Phylip dnacomp program output
                                  file

   Additional (Optional) qualifiers (* if not always prompted):
   -outgrno            integer    [0] Species number to use as outgroup
                                  (Integer 0 or more)
*  -njumble            integer    [0] Number of times to randomise (Integer 0
                                  or more)
*  -seed               integer    [1] Random number seed between 1 and 32767
                                  (must be odd) (Integer from 1 to 32767)
   -[no]trout          toggle     [Y] Write out trees to tree file
*  -outtreefile        outfile    [*.fdnacomp] Phylip tree output file
                                  (optional)
   -printdata          boolean    [N] Print data at start of run
   -[no]progress       boolean    [Y] Print indications of progress of run
   -[no]treeprint      boolean    [Y] Print out tree
   -stepbox            boolean    [N] Print steps & compatibility at sites
   -ancseq             boolean    [N] Print sequences at all nodes of tree

   Advanced (Unprompted) qualifiers: (none)
   Associated qualifiers:

   "-sequence" associated qualifiers
   -sbegin1            integer    Start of each sequence to be used
   -send1              integer    End of each sequence to be used
   -sreverse1          boolean    Reverse (if DNA)
   -sask1              boolean    Ask for begin/end/reverse
   -snucleotide1       boolean    Sequence is nucleotide
   -sprotein1          boolean    Sequence is protein
   -slower1            boolean    Make lower case
   -supper1            boolean    Make upper case
   -scircular1         boolean    Sequence is circular
   -sformat1           string     Input sequence format
   -iquery1            string     Input query fields or ID list
   -ioffset1           integer    Input start position offset
   -sdbname1           string     Database name
   -sid1               string     Entryname
   -ufo1               string     UFO features
   -fformat1           string     Features format
   -fopenfile1         string     Features file name

   "-outfile" associated qualifiers
   -odirectory3        string     Output directory

   "-outtreefile" associated qualifiers
   -odirectory         string     Output directory

   General qualifiers:
   -auto               boolean    Turn off prompts
   -stdout             boolean    Write first file to standard output
   -filter             boolean    Read first file from standard input, write
                                  first file to standard output
   -options            boolean    Prompt for standard and additional values
   -debug              boolean    Write debug output to program.dbg
   -verbose            boolean    Report some/full command line options
   -help               boolean    Report command line options and exit. More
                                  information on associated and general
                                  qualifiers can be found with -help -verbose
   -warning            boolean    Report warnings
   -error              boolean    Report errors
   -fatal              boolean    Report fatal errors
   -die                boolean    Report dying program messages
   -version            boolean    Report version number and exit

Qualifier Type Description Allowed values Default

Standard (Mandatory) qualifiers

[-sequence]
(Parameter 1) seqsetall File containing one or more sequence alignments Readable sets of sequences Required

[-intreefile]
(Parameter 2) tree Phylip tree file (optional) Phylogenetic tree

-weights properties Phylip weights file (optional) Property value(s)

[-outfile]
(Parameter 3) outfile Phylip dnacomp program output file Output file <*>.fdnacomp

Additional (Optional) qualifiers

-outgrno integer Species number to use as outgroup Integer 0 or more 0

-njumble integer Number of times to randomise Integer 0 or more 0

-seed integer Random number seed between 1 and 32767 (must be odd) Integer from 1 to 32767 1

-[no]trout toggle Write out trees to tree file Toggle value Yes/No Yes

-outtreefile outfile Phylip tree output file (optional) Output file <*>.fdnacomp

-printdata boolean Print data at start of run Boolean value Yes/No No

-[no]progress boolean Print indications of progress of run Boolean value Yes/No Yes

-[no]treeprint boolean Print out tree Boolean value Yes/No Yes

-stepbox boolean Print steps & compatibility at sites Boolean value Yes/No No

-ancseq boolean Print sequences at all nodes of tree Boolean value Yes/No No

Advanced (Unprompted) qualifiers

(none)

Associated qualifiers

"-sequence" associated seqsetall qualifiers

-sbegin1
-sbegin_sequence integer Start of each sequence to be used Any integer value 0

-send1
-send_sequence integer End of each sequence to be used Any integer value 0

-sreverse1
-sreverse_sequence boolean Reverse (if DNA) Boolean value Yes/No N

-sask1
-sask_sequence boolean Ask for begin/end/reverse Boolean value Yes/No N

-snucleotide1
-snucleotide_sequence boolean Sequence is nucleotide Boolean value Yes/No N

-sprotein1
-sprotein_sequence boolean Sequence is protein Boolean value Yes/No N

-slower1
-slower_sequence boolean Make lower case Boolean value Yes/No N

-supper1
-supper_sequence boolean Make upper case Boolean value Yes/No N

-scircular1
-scircular_sequence boolean Sequence is circular Boolean value Yes/No N

-sformat1
-sformat_sequence string Input sequence format Any string

-iquery1
-iquery_sequence string Input query fields or ID list Any string

-ioffset1
-ioffset_sequence integer Input start position offset Any integer value 0

-sdbname1
-sdbname_sequence string Database name Any string

-sid1
-sid_sequence string Entryname Any string

-ufo1
-ufo_sequence string UFO features Any string

-fformat1
-fformat_sequence string Features format Any string

-fopenfile1
-fopenfile_sequence string Features file name Any string

"-outfile" associated outfile qualifiers

-odirectory3
-odirectory_outfile string Output directory Any string

"-outtreefile" associated outfile qualifiers

-odirectory string Output directory Any string

General qualifiers

-auto boolean Turn off prompts Boolean value Yes/No N

-stdout boolean Write first file to standard output Boolean value Yes/No N

-filter boolean Read first file from standard input, write first file to standard output Boolean value Yes/No N

-options boolean Prompt for standard and additional values Boolean value Yes/No N

-debug boolean Write debug output to program.dbg Boolean value Yes/No N

-verbose boolean Report some/full command line options Boolean value Yes/No Y

-help boolean Report command line options and exit. More information on associated and general qualifiers can be found with -help -verbose Boolean value Yes/No N

-warning boolean Report warnings Boolean value Yes/No Y

-error boolean Report errors Boolean value Yes/No Y

-fatal boolean Report fatal errors Boolean value Yes/No Y

-die boolean Report dying program messages Boolean value Yes/No Y

-version boolean Report version number and exit Boolean value Yes/No N

Qualifier	Type	Description	Allowed values	Default
Standard (Mandatory) qualifiers
[-sequence] (Parameter 1)	seqsetall	File containing one or more sequence alignments	Readable sets of sequences	Required
[-intreefile] (Parameter 2)	tree	Phylip tree file (optional)	Phylogenetic tree
-weights	properties	Phylip weights file (optional)	Property value(s)
[-outfile] (Parameter 3)	outfile	Phylip dnacomp program output file	Output file	<>*.fdnacomp
Additional (Optional) qualifiers
-outgrno	integer	Species number to use as outgroup	Integer 0 or more	0
-njumble	integer	Number of times to randomise	Integer 0 or more	0
-seed	integer	Random number seed between 1 and 32767 (must be odd)	Integer from 1 to 32767	1
-[no]trout	toggle	Write out trees to tree file	Toggle value Yes/No	Yes
-outtreefile	outfile	Phylip tree output file (optional)	Output file	<>*.fdnacomp
-printdata	boolean	Print data at start of run	Boolean value Yes/No	No
-[no]progress	boolean	Print indications of progress of run	Boolean value Yes/No	Yes
-[no]treeprint	boolean	Print out tree	Boolean value Yes/No	Yes
-stepbox	boolean	Print steps & compatibility at sites	Boolean value Yes/No	No
-ancseq	boolean	Print sequences at all nodes of tree	Boolean value Yes/No	No
Advanced (Unprompted) qualifiers
(none)
Associated qualifiers
"-sequence" associated seqsetall qualifiers
-sbegin1 -sbegin_sequence	integer	Start of each sequence to be used	Any integer value	0
-send1 -send_sequence	integer	End of each sequence to be used	Any integer value	0
-sreverse1 -sreverse_sequence	boolean	Reverse (if DNA)	Boolean value Yes/No	N
-sask1 -sask_sequence	boolean	Ask for begin/end/reverse	Boolean value Yes/No	N
-snucleotide1 -snucleotide_sequence	boolean	Sequence is nucleotide	Boolean value Yes/No	N
-sprotein1 -sprotein_sequence	boolean	Sequence is protein	Boolean value Yes/No	N
-slower1 -slower_sequence	boolean	Make lower case	Boolean value Yes/No	N
-supper1 -supper_sequence	boolean	Make upper case	Boolean value Yes/No	N
-scircular1 -scircular_sequence	boolean	Sequence is circular	Boolean value Yes/No	N
-sformat1 -sformat_sequence	string	Input sequence format	Any string
-iquery1 -iquery_sequence	string	Input query fields or ID list	Any string
-ioffset1 -ioffset_sequence	integer	Input start position offset	Any integer value	0
-sdbname1 -sdbname_sequence	string	Database name	Any string
-sid1 -sid_sequence	string	Entryname	Any string
-ufo1 -ufo_sequence	string	UFO features	Any string
-fformat1 -fformat_sequence	string	Features format	Any string
-fopenfile1 -fopenfile_sequence	string	Features file name	Any string
"-outfile" associated outfile qualifiers
-odirectory3 -odirectory_outfile	string	Output directory	Any string
"-outtreefile" associated outfile qualifiers
-odirectory	string	Output directory	Any string
General qualifiers
-auto	boolean	Turn off prompts	Boolean value Yes/No	N
-stdout	boolean	Write first file to standard output	Boolean value Yes/No	N
-filter	boolean	Read first file from standard input, write first file to standard output	Boolean value Yes/No	N
-options	boolean	Prompt for standard and additional values	Boolean value Yes/No	N
-debug	boolean	Write debug output to program.dbg	Boolean value Yes/No	N
-verbose	boolean	Report some/full command line options	Boolean value Yes/No	Y
-help	boolean	Report command line options and exit. More information on associated and general qualifiers can be found with -help -verbose	Boolean value Yes/No	N
-warning	boolean	Report warnings	Boolean value Yes/No	Y
-error	boolean	Report errors	Boolean value Yes/No	Y
-fatal	boolean	Report fatal errors	Boolean value Yes/No	Y
-die	boolean	Report dying program messages	Boolean value Yes/No	Y
-version	boolean	Report version number and exit	Boolean value Yes/No	N

Input file format

fdnacomp reads any normal sequence USAs.

Input files for usage example

File: dnacomp.dat

    5   13
Alpha     AACGUGGCCAAAU
Beta      AAGGUCGCCAAAC
Gamma     CAUUUCGUCACAA
Delta     GGUAUUUCGGCCU
Epsilon   GGGAUCUCGGCCC

Input files for usage example 2

File: dnacomptree.dat

((((Epsilon,Delta),Gamma),Beta),Alpha);

Output file format

fdnacomp output is standard: if option 1 is toggled on, the data is printed out, with the convention that "." means "the same as in the first species". Then comes a list of equally parsimonious trees, and (if option 2 is toggled on) a table of the number of changes of state required in each character. If option 5 is toggled on, a table is printed out after each tree, showing for each branch whether there are known to be changes in the branch, and what the states are inferred to have been at the top end of the branch. If the inferred state is a "?" or one of the IUB ambiguity symbols, there will be multiple equally-parsimonious assignments of states; the user must work these out for themselves by hand. A "?" in the reconstructed states means that in addition to one or more bases, a gap may or may not be present. If option 6 is left in its default state the trees found will be written to a tree file, so that they are available to be used in other programs. If the program finds multiple trees tied for best, all of these are written out onto the output tree file. Each is followed by a numerical weight in square brackets (such as [0.25000]). This is needed when we use the trees to make a consensus tree of the results of bootstrapping or jackknifing, to avoid overrepresenting replicates that find many tied trees.

Output files for usage example

File: dnacomp.fdnacomp


DNA compatibility algorithm, version 3.69.650

 5 species,  13  sites

Name            Sequences
----            ---------

Alpha        AACGTGGCCA AAT
Beta         AAGGTCGCCA AAC
Gamma        CATTTCGTCA CAA
Delta        GGTATTTCGG CCT
Epsilon      GGGATCTCGG CCC



One most parsimonious tree found:




           +--Epsilon   
        +--4  
     +--3  +--Delta     
     !  !  
  +--2  +-----Gamma     
  !  !  
  1  +--------Beta      
  !  
  +-----------Alpha     

  remember: this is an unrooted tree!


total number of compatible sites is       11.0

steps in each site:
         0   1   2   3   4   5   6   7   8   9
     *-----------------------------------------
    0|       2   1   3   2   0   2   1   1   1
   10|   1   1   1   3                        

 compatibility (Y or N) of each site with this tree:

      0123456789
     *----------
   0 ! YYNYYYYYY
  10 !YYYN      

From    To     Any Steps?    State at upper node
                            
          1                AABGTSGCCA AAY
   1      2        maybe   AABGTCGCCA AAY
   2      3         yes    VAKDTCGCCA CAY
   3      4         yes    GGKATCTCGG CCY
   4   Epsilon     maybe   GGGATCTCGG CCC
   4   Delta        yes    GGTATTTCGG CCT
   3   Gamma        yes    CATTTCGTCA CAA
   2   Beta        maybe   AAGGTCGCCA AAC
   1   Alpha       maybe   AACGTGGCCA AAT

File: dnacomp.treefile

((((Epsilon,Delta),Gamma),Beta),Alpha);

Output files for usage example 2

File: dnacomp.fdnacomp


DNA compatibility algorithm, version 3.69.650

User-defined tree:



           +--Epsilon   
        +--4  
     +--3  +--Delta     
     !  !  
  +--2  +-----Gamma     
  !  !  
  1  +--------Beta      
  !  
  +-----------Alpha     

  remember: this is an unrooted tree!


total number of compatible sites is       11.0

Data files

None

Notes

None.

References

None.

Warnings

None.

Diagnostic Error Messages

None.

Exit status

It always exits with status 0.

Known bugs

None.

Program name	Description
distmat	Create a distance matrix from a multiple sequence alignment
ednacomp	DNA compatibility algorithm
ednadist	Nucleic acid sequence distance matrix program
ednainvar	Nucleic acid sequence invariants method
ednaml	Phylogenies from nucleic acid maximum likelihood
ednamlk	Phylogenies from nucleic acid maximum likelihood with clock
ednapars	DNA parsimony algorithm
ednapenny	Penny algorithm for DNA
eprotdist	Protein distance algorithm
eprotpars	Protein parsimony algorithm
erestml	Restriction site maximum likelihood method
eseqboot	Bootstrapped sequences algorithm
fdiscboot	Bootstrapped discrete sites algorithm
fdnadist	Nucleic acid sequence distance matrix program
fdnainvar	Nucleic acid sequence invariants method
fdnaml	Estimate nucleotide phylogeny by maximum likelihood
fdnamlk	Estimates nucleotide phylogeny by maximum likelihood
fdnamove	Interactive DNA parsimony
fdnapars	DNA parsimony algorithm
fdnapenny	Penny algorithm for DNA
fdolmove	Interactive Dollo or polymorphism parsimony
ffreqboot	Bootstrapped genetic frequencies algorithm
fproml	Protein phylogeny by maximum likelihood
fpromlk	Protein phylogeny by maximum likelihood
fprotdist	Protein distance algorithm
fprotpars	Protein parsimony algorithm
frestboot	Bootstrapped restriction sites algorithm
frestdist	Calculate distance matrix from restriction sites or fragments
frestml	Restriction site maximum likelihood method
fseqboot	Bootstrapped sequences algorithm
fseqbootall	Bootstrapped sequences algorithm

Author(s)

This program is an EMBOSS conversion of a program written by Joe Felsenstein as part of his PHYLIP package.

History

Written (2004) - Joe Felsenstein, University of Washington.

Converted (August 2004) to an EMBASSY program by the EMBOSS team.

Target users

This program is intended to be used by everyone and everything, from naive users to embedded scripts.

Comments

None

Wiki

Function

Description

Algorithm

Usage

Command line arguments

Input file format

Input files for usage example

File: dnacomp.dat

Input files for usage example 2

File: dnacomptree.dat

Output file format

Output files for usage example

File: dnacomp.fdnacomp

File: dnacomp.treefile

Output files for usage example 2

File: dnacomp.fdnacomp

Data files

Notes

References

Warnings

Diagnostic Error Messages

Exit status

Known bugs

See also

Author(s)

History

Target users

Comments