Talk:Release 1.6

From BioPerl
Jump to: navigation, search

Contents

Introduction

The following issues/bugs need to be addressed prior to 1.6. Please voice opinions on the discussion page (this will be edited to reflect changes).

Following is the general priority list for the BioPerl 1.6 release. This list is neither meant to be comprehensive nor does it represent everything that must be completed for the next immediate release, but indicates issues that need to be addressed. The end game is to start setting and prioritizing goals for 1.6 and future releases. Feel free to comment where needed. If there are any additional concerns or items now is the time to mention them so they can be addressed.

Also, if anyone wants to work on certain items, please indicate so by attaching your signature (--~~~~). Items 'locked down' for the release series (along with relevant devs working on them) will be posted to the main release 1.6 page (accessible only to wiki sysops).

Discussion will continue for two weeks starting Nov. 21, allowing some time for the Thanksgiving holiday (extensions are possible if more issues need to be hashed out). Release priorities will be set soon after with the goal of releasing 1.6 within a reasonable time (2-4 weeks if possible).

Bio::Graphics and Splitting BioPerl

NOTE: This is essentially complete. The only thing left to consider is adding some kind of post-installation option for Bio::Graphics (if needed).

  1. We are planning on splitting up BioPerl core into smaller (more manageable) subdistributions; this will mainly occur after 1.6. However, Gbrowse-related modules (Bio::Graphics, Bio::DB::SeqFeature, Bio::DB::GFF) are undergoing rapid development, much more than the rest of the BioPerl modules. Therefore I suggest we test the waters (so to speak) and split those off prior to the 1.6 release into their own distribution. This will allow Lincoln and the Gbrowse crew to proceed with development on those bits of bioperl independently; the only requirement would be a compatible BioPerl release, preferably in CPAN. If needed this can be tested in a release candidate prior to a full release.
    1. Scott Cain indicating a split after BioPerl would just overwrite the outdated modules.
    2. Chris's response - how do we handle fixes on the 1.6 branch after a split?
    3. Bio::Graphics is now [split out]. --Chris Fields 04:27, 9 December 2008 (UTC)
  2. If we split off Bio::Graphics and related, we should sort out versioning schemes issues for these subdistributions prior to splitting them off. Should they be completely independent versions or be tied in to core (i.e. be 1.6.x, which would be expected to have a requirement for bioperl 1.6)?
    1. TBD, but may be a minor issue.
  3. At this point I am considering the other BioPerl distributions (db, network, pedigree, etc) as independent subdistributions, similar to what may happen with Bio::Graphics etc. Therefore, these will also have their own independent release schedule. Once 1.6 is released, we can work on getting the other bioperl-* released to CPAN in fairly rapid manner so they all correspond to core 1.6, but from that point on they can follow a separate release schedule.
  4. Should we add an option to install subdistributions in the core build script?
    1. The idea may be to use a Bundle which emulates the former bioperl installation (would install all bioperl-related packages or select ones).

Bugs and Bugzilla

NOTE: essentially complete. Remaining bugs pertain more to bioperl-run that bioperl-live or are minor (and will probably be pushed to 1.6.x).

  1. Identify outstanding bugs which need to be addressed prior to release (I have accepted a couple myself which I've been working on), or should be addressed sometime in the 1.6 release series (i.e. fixed in a point release if there are no API changes). I will be adding a table to the wiki page noted above grouping related bugs with notes.
  2. Add any outstanding issues to bugzilla for tracking.
  3. We need to consider merging the enhancement requests and the project priority list in some meaningful way (either moving everything to the wiki, or organizing the enhancement requests for various Bio::* modules in bugzilla).
  4. To add/remove: log in to BugZilla and change the target. Only true bugs will have priority (no enhancement requests).
  5. Add more milestones, components, etc. Suggestions?
Failed to load RSS feed from http://bugzilla.open-bio.org/buglist.cgi?bug_file_loc=&bug_file_loc_type=allwordssubstr&bug_id=&bug_severity=blocker&bug_severity=critical&bug_severity=major&bug_severity=normal&bug_severity=minor&bug_severity=trivial&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bugidtype=include&chfieldfrom=&chfieldto=Now&chfieldvalue=&email1=&email2=&emailassigned_to1=1&emailassigned_to2=1&emailcc2=1&emailqa_contact2=1&emailreporter2=1&emailtype1=substring&emailtype2=substring&field-1-0-0=product&field-1-1-0=target_milestone&field-1-2-0=bug_status&field-1-3-0=bug_severity&field0-0-0=noop&keywords=&keywords_type=allwords&long_desc=&long_desc_type=substring&product=BioPerl&query_format=advanced&remaction=&short_desc=&short_desc_type=allwordssubstr&status_whiteboard=&status_whiteboard_type=allwordssubstr&target_milestone=1.6%20release&type-1-0-0=anyexact&type-1-1-0=anyexact&type-1-2-0=anyexact&type-1-3-0=anyexact&type0-0-0=noop&value-1-0-0=BioPerl&value-1-1-0=1.6%20release&value-1-2-0=NEW%2CASSIGNED%2CREOPENED&value-1-3-0=blocker%2Ccritical%2Cmajor%2Cnormal%2Cminor%2Ctrivial&value0-0-0=&votes=&title=Bug%20List&ctype=atom!

Test suite

NOTE : Test suite is reorganized and BioPerlTest is now Bio::Root::Test. We may eventually start a similar reorganization of bioperl-run tests after core 1.6 is released. Will make test coverage more prominent so devs can find it easily.

  1. Though it isn't critical for the release, we should attempt some simple reorganization of the test suite to deal with the distribution split mentioned in (1) above. A significant amount can easily be accomplished by the 1.6 release, but we need to make sure no tests are left out or repeated. I have already done some simplification of the SearchIO tests by splitting them up by parser; we can do something similar for Bio::SeqIO, Bio::AlignIO, etc. Dave Messina has already taken up splitting and consolidating SeqIO tests, so any other volunteers other relevant bits would be appreciated. Following is a simple list so we have some to work with (again, this may change depending on comments):
  2. Tests specific for parsers/plugins: make a subdirectory in t/ to group related tests (ex: SeqIO/genbank.t, AlignIO/stockholm.t). I left the plugin as lowercase here, which is the naming convention we are using for the module names.
  3. Similarly, if modules belong into one collective group, they can follow a similar Group/Class.t (Tools/Genewise.t). Here the module being tested is uppercase (not a plugin).
  4. How far do we want to take this? Should Location.t be split up by the various Bio::Location objects into Location_*.t (and similarly, Bio::Annotation, Bio::SeqFeature, etc)? Again, not a blocker for 1.6, should be fairly easy to do, but this will increase the number of test files quite significantly.
  5. We have a set of BioPerl-specific test functions that Sendu graciously set up in the bioperl 1.5.2 release. We should actually wrap these into core (Bio::Root::Test or similar) and have them become generally available for any Bio::* modules, either as exported methods in a package or as a full-on class. Similarly, we should add a test file for the BioPerl-specific test functions using Test::More functions only. This can possibly wait until 1.7, but it shouldn't be too hard to implement.
  6. Test coverage is already in place (thanks Mauricio, Nathan, Sendu, and everyone else involved)! What else needs to be done?
    1. This has been setup and has been working for a few weeks now: http://bioperl.org/test-coverage/ --Mauricio 04:24, 18 November 2008 (UTC)

Developer vs. Stable releases

NOTE : Adding a small bit to INSTALL docs about future releases of BioPerl having a separate developer release.

  1. After 1.6, no more alternating of 'developer' and 'stable' releases. The designation is highly misleading, particularly when one considers how stable current subversion code is compared to the 'stable' 1.4 release in CPAN.
  2. By consequence of losing 'dev' releases, we will need to to etch out a place so developers can test new code, place untested/unsupported code which may still be useful, etc. bioperl-experimental is currently designated as the sandbox for Perl6 development, so I suggest bioperl-dev (bioperl-here-thar-be-dragoons is too long).
    1. Code which has inadequate test coverage (or no tests at all) or isn't actively maintained or supported can be split off into a separate 'dev' package unless absolutely necessary to the core. Once those 'dev' packages reach full test coverage and have decent enough docs they can graduate to 'core', 'run', or whatever. --Chris Fields 10:04, 23 August 2007 (EDT)
    2. Agree. And all new code should first go there until its coverage is 100%. --Senduran 10:10, 23 August 2007 (EDT)
  3. Significant changes to modules already present (API changes, for instance) should be run on a branch to the relevant distribution. Accepted revisions can be merged back into trunk with a subsequent minor release.
    1. Delete branches after merges? This is fairly common practice, but we seem to keep all our branches.

Point releases vs minor releases

  1. Release regular bug fixes (no API changes) as point releases.
  2. Small API changes require minor releases. Related: what will the module split version be (1.7? 2.0?)
  3. Deprecation of modules is decided by the core devs. If the module is widely used (i.e. Bio::Species) we should go through a routine deprecation cycle prior to removing the module from a distribution. Some modules (for instance, DB modules which no longer work due to changes in remote access such as XEMBL) can be immediately deprecated and removed in the next release.

General Discussion

  • Chris, for starters, I can take care of splitting out the SeqIO tests. Is the below list the to-dos for 1.6 to ship? I saw you doing a bugzilla sweep yesterday; perhaps you could add those which you want done for 1.6 here so devs can pick them up -Dave Messina 15:47, 21 November 2008 (UTC)
  • I have added an RSS feed for bugs slated for 1.6 above. We need to determine which bugs should be targeted for 1.6.0, 1.6.x, and others which be pushed off to 1.7. I'll look into Bugzilla to see if I can add more refined targets. --Chris Fields 22:32, 21 November 2008 (UTC)
  • I've gone through all the bugs in the above RSS feed, put them in the table below, and marked them with comments on their status as best as I can tell and my thoughts on which release they should be targeted for. Chris and anyone else with a point of view on these, please add your comments. -Dave Messina 21:30, 27 November 2008 (UTC)

To-Dos for core 1.6 (general)

Task Assigned to Completed? Notes
decide whether to move Bio::Graphics and Gbrowse-related modules out of BioPerl Lincoln Stein Yes A split off of Bio::Graphics is complete and in CPAN
if yes, move Gbrowse out of BioPerl Lincoln Stein Yes The plan is to split off Bio::Graphics first and genericize Bio::Graphics::FeatureBase (Bio::SeqFeature::Lite, maybe). Lincoln is working on this.
split out SearchIO tests Chris Fields Yes
split out SeqIO tests Dave Messina Yes
split out AlignIO tests Chris Yes Done. --Chris Fields 20:29, 4 December 2008 (UTC)
split out and reorganize everthing! Heikki Yes May need further splitting up, but the major reorganization is done. (Yay Heikki!)
Move BioPerlTest into core (Bio::Root::Test), and have all bioperl-* tests use that Sendu Yes tests now switched over
Move ModulePerlBuild into core (Bio::Root::Build), all bioperl-* Build.PL refer to that Sendu Yes Build.PL now uses this; will switch others over

To-Dos for core 1.6 (bugs)

Task Assigned to Completed? Notes
LocatableSeq translated nuc bug Chris yes Issue #2630
seq_inds refactor Chris yes Issue #2663
SeqFeature::Annotation requires network access Chris pushed to 1.6.x Issue #2513 - I vote for this to be marked as an enhancement and bumped to 1.6.x point release. It's about reimplementing in a smarter way -- the existing code does what it was designed to do.
This one's a particularly annoying bug (slows network tests down considerably). --Chris Fields 20:26, 29 November 2008 (UTC)
fix nightly build script Chris pushed to 1.6.x Issue #2545 - I vote for this to be bumped to 1.6.x point release. It's about the bioperl website and unrelated to releasing 1.6.
It would be nice to get running, but isn't necessary for a release (it just packs up a nightly svn repo tarballs) --Chris Fields 23:29, 28 November 2008 (UTC)
HTMLResultWriter improvements Chris pushed to 1.6.x Issue #2439 - Chris has marked this as a hopeful for 1.6. I'd argue it's more of an enhancement and should be pushed to 1.6.x point.
Agreed. --Chris Fields 23:29, 28 November 2008 (UTC)
Bio::Tree::TreeFunctionsI reroot mucks up bootstrap values Chris pushed to 1.6.x Issue #2456 - Sendu commented on the bug that it's a nontrivial fix, which imo means it needs to be pushed back to 1.6.x point unless someone can commit the time to doing this now.
incorrect "undefined sub-sequence" errors from length_aln Chris maybe for 1.6 Issue #2476 - length_aln relies on tile_hsps, which is experimental code, known to be brittle, and probably in need of refactoring. I argue this is not critical for 1.6 release and should be bumped to 1.6.x.
Possible fix posted by Mark Jensen, code under revision --Chris Fields 03:37, 9 December 2008 (UTC)
paml4 mlc file compatibility Chris pushed to 1.6.x Issue #2482 - tough to keep up with changes in PAML's format. It's be great to get this one in for 1.6, but since this isn't really critical (in my opinion), I'd like us to push this off to 1.6.x unless someone can devote the time in the next week or two.
Pushing to 1.7; I think this will require a refactor to DTRT so may be an API change. --Chris Fields 23:29, 28 November 2008 (UTC)
Bio::Species memory leak Chris attempt for 1.6 Issue #2594 - done. Not yet committed (I think?)
Checked in code removes requirement for weaken/is_weak, but the memory leak persists; likely an issue with Bio::Tree::Tree or Bio::Taxon. --Chris Fields 03:37, 9 December 2008 (UTC)
PAML::baseml polishing Chris pushed to 1.7 Issue #2347 - already pushed to 1.7.
Bio::DB::Flat::BDB::fasta builds incorrect index Brian O. Yes Issue #2336 - appears to be fixed.
BDB config data Brian O. bumped into OBDA bug queue Issue #2337 - this is a OBDA spec issue that has been bumped to the new OBDA bug queue in Bugzilla. --Chris Fields 03:43, 9 December 2008 (UTC)
bequeath and bequest fields to PrimarySeq Chris pushing to 1.6.x or 1.7 Issue #2673 - Patch and test done. For 1.6 release (if approved).
This is a good solution that needs to be implemented, but not in a rush (and not for this release). Maybe 1.6.x if implementation works well. --Chris Fields 03:43, 9 December 2008 (UTC)
cleanup stockholm alignment output Chris Yes Issue #2450 - Done?
Done. --Chris Fields 03:43, 9 December 2008 (UTC)
pi calc in PopGen::Statistics pushing to 1.6.x Issue #2492 - Awaiting comment from Jason . Looks like done and should be perhaps recategorized as future enhancement.
investigate warnings from tests for SeqIO::chadoxml, SeqIO::genbank (really Location::Fuzzy), SeqIO::phd. Dave yes I have partially fixed some issues. Still needs more work. --Chris Fields 23:29, 28 November 2008 (UTC)

Tests are passing as of now (thanks Chris!), so I'm marking this as done. --Dave Messina 20:13, 4 December 2008 (UTC)

To-Dos for bioperl-run 1.6 (bugs and general)

Task Assigned to Completed? Notes
bioperl-run test errors Chris Maybe? Issue #2514 - a few minor errors cropping up now with the bioperl-run test suite. Should be done for 1.6 release, I think.
Agreed. --Chris Fields 23:29, 28 November 2008 (UTC)

To-Dos for bioperl-db 1.6 (bugs and general)

Task Assigned to Completed? Notes
bioperl-db NCBI download crash pushed to 1.6.x Issue #2213 - I vote for this to be downgraded from critical as it's no longer a crashing bug and to be bumped to 1.6.x point release.
same seq, diff namespace bioperl-db error pushed to 1.6.x Issue #2280 - This should be grouped with Issue #2213 and Issue #2275 as part of a closer look at BioPerl's relationship with BioSQL and bumped to 1.6.x point release.
Genus species storage errors with bioperl-db pushed to 1.6.x Issue #2092 - belongs with the other bioperl-db/BioSQL issues (Issue #2280, Issue #2213, Issue #2275, Issue #2472, Issue #2474). I argue for bump back to 1.6.x point release.
We should fix this minimally and work on a Taxon-based system compatible with other Bio* implementations. --Chris Fields 23:29, 28 November 2008 (UTC)
GO_upload script errors pushed to 1.6.x Issue #2275 - This should be grouped with Issue #2213 as part of a closer look at BioPerl's relationship with BioSQL and bumped to 1.6.x point release.
BioSQL::SeqAdaptor::find_by_query needs cast in pgSQL pushed to 1.6.x Issue #2472 - this looks like a fairly trivial fix, but we'd want to make sure implementing an explicit cast is kosher with other SQL implementations besides pgSQL.This is minor and probably can be grouped with the other BioSQL stuff (Issue #2280, Issue #2213, Issue #2275) under 1.6.x point.
load_seqdatabase.pl drops some UniProt entries pushed to 1.6.x Issue #2474 - Another BioSQL issue which I argue should not hold up 1.6 unless someone knowledgeable with its workings has the time for this. Group with Issue #2280, Issue #2213, Issue #2275, Issue #2472.
update ontology test data Yes (essentially) Issue #2573 - Fix already committed, jus need to update tests with new ontology data. Probably doable for 1.6 release.

To-Dos for bioperl-network 1.6 (bugs and general)

Nothing of note. Should be a straightforward release.

To-Dos for bioperl-pedigree 1.6 (bugs and general)

Nothing of note. Should be a straightforward release.

To-Dos for bioperl-ext 1.6 (bugs and general)

A true 1.6 release of bioperl-ext is unlikely, but it may be the focus of a future bioperl release.

Task Assigned to Completed? Notes
dpAlign external C code Linux-specific bug yes Issue #2384 - Chris' comment on the bug already indicated this should be pushed to 1.7. I agree and already updated bugzilla to reflect that.

Future Releases

The following are now pushed to later releases.

  • Split BioPerl into more manageable subdistributions, making core as lean as possible.

The below are much easier to accomplish once we split bioperl into set subdistributions, so should wait until that is accomplished:

  • Determine who is available to maintain modules or distributions.
  • Along those lines, assign orphaned modules to new maintainers if possible
  • Each maintainer, for their set of modules:
    • Uses Devel::Cover (with Pod::Coverage) via Module::Build to:
      • Check and complete POD documentation for all methods
      • Improve test coverage: try to ensure that all methods are fully tested in t/module.t
    • Reports which modules are complete and which are not
  • For incomplete or unmaintained modules introduced since 1.4, move, remove, or revert

The following may require some API reconfiguration or lots of work:

  • Refactor Bio::LocatableSeq and Bio::SimpleAlign.
    • I think LocatableSeq should actually be a specialized Bio::SeqI. It would store it's RangeI information as a Bio::LocationI in a full-length Bio::SeqFeature::Generic (tagged as 'source'). Other attributes could be stored as annotation or SeqFeatures as needed. I also would like to derive a better way to deal between coordinate systems (I have added a few methods recently along those lines, but they are really hacky until we come up with something better). --Chris Fields 01:38, 23 November 2008 (UTC)
  • Implement trunc(), at least for Bio::Seq
Personal tools
Namespaces
Variants
Actions
Main Links
documentation
community
development
Toolbox