OEP-21: Deprecation and Removal

OEP OEP-21
Title Deprecation and Removal
Last Modified 2018-10-26
Authors Greg Sham <gsham@edx.org>, Nimisha Asthagiri <nimisha@edx.org> Diana Huang <dkh@edx.org>
Arbiter Matt Tuchfarber <mtuchfarber@edx.org>
Status Accepted
Type Process
Created 2018-05-18
Review Period 2018-06-06 - 2018-06-20

Abstract

“Remove dead code wherever possible. It gets in the way and slows you down.” [OREILLY]

This document defines a formal process for proposing, communicating, deprecating and removing legacy and unneeded code in the open edX platform.

_images/dead-code.png

Motivation

Without a clearly defined and agreed upon process for removing legacy code, open edX developers will hesitate to do so. As a result, the platform will have redundant implementations and dormant features that over time will slow down development and developer onboarding.

This is a problem shared by the software engineering industry and many have learned the benefits of investing in code removal:

  • It is undeniable that unnecessary code, like any other code, requires maintenance over time. It costs time and money. [OREILLY]
  • Extra code also makes it harder to learn the project, and requires extra understanding and navigating. [OREILLY]
  • It is harder to refactor, simplify, or optimize your program when it is bogged down by zombie code. [OREILLY]
  • There is a danger that someone inadvertently involves the ‘dormant’ code and introduces bugs. [SO]
  • Dead code makes the runtime footprint larger than it needs to be. [INFOQ]
  • Dead code discourages a culture of treating the software as soft, and therefore always open to revision and improvement. [INFOQ]

Given the above, we define a common language and process to remove unneeded code. That code may be superseded by a new implementation or may be deemed no longer necessary and supported. Following this process guarantees alignment across all stakeholders and full execution.

Specification

This section explains each step in the process to remove a technology from the Open edX platform.

Process States

A diagram that shows the state flow transitions. The process starts in the Proposed state and goes through the *Communicated*, *Accepted*, *Deprecated*, *Removing*, and *Removed* states. If the proposal isn't *Accepted*, the state transitions directly from *Proposed* to *Abandoned*.

As shown in the diagram above, the code removal process flows through the following states.

  • Proposed - The starting state when removal is considered, analyzed, and documented.
  • Communicated - The proposal is announced to the general Open edX community via the broadcast mechanisms defined below. Specific notifications are sent to all identified stakeholders. The corresponding JIRA ticket is updated with community feedback until the target Accepted date.
  • Accepted - The proposal is officially accepted after addressing community feedback and aligning on a timeline.
  • Deprecated - If required, the code is updated and marked as deprecated.
  • Removing - A team has started removing the code.
  • Removed - The code removal process is fully completed.
  • Abandoned - The code removal process was aborted and will not complete.

The next sections go through the details of each state and transition.

Proposed

Do you think a piece of the codebase should be removed? If you even asked that question, chances are likely that it probably should be.

When to remove?

Here are a few common cases for code removal:

  • There is now a new implementation that replaces the old implementation. If so, the old implementation should be deprecated and then removed in favor of the new.
  • Static analysis or runtime analysis shows that the code is never executed on your open edX instance. If so, it should be removed (if no one in the community requires it) or should become a pluggable extension since it’s not core to all instances.
  • There is a legacy feature that never really saw the light of the day or was adopted by very few users. If so, this should be confirmed by usage analysis and then removed.

What to remove?

Here are a few technologies that are commonly removed:

  • User features, including their APIs, frontend, and backend implementations
  • Modular components, such as Django apps, Frontend apps, XBlocks
  • Technologies, such as CoffeeScript, outdated frameworks
  • Feature toggles used for temporary rollout and testing
  • Interfaces, such as REST APIs and Plugin APIs

Analyze

When proposing a removal, consider the following analysis:

  • Usage - Which users and services are currently using the code proposed for removal on your own open edX instance? Perform a quick search across the edX codebase to gauge the level of impact and identify potential stakeholders. https://github.com/search?q=org%3Aedx+sample&type=Code
  • Replacement - What, if any, is a viable replacement for the code being removed?
  • Migration path - If there is existing high usage in the community, what is a viable automated migration path from the deprecated code to the removed code?
  • Deprecation - Based on expected usage and effort to migrate, for how long should the deprecation period be?

Timeline

When determining target dates to propose for the removal process, consider that it will vary depending on team resources and the technology being removed. The importance of removal (as described in Motivation) should be communicated with all team members so the removal can be prioritized and completed in a timely manner. A suggested timeline is shown in the diagram below, which considers the timing of the next Open edX named release.

A diagram that suggests having a 2 week time period between the *Proposed* and *Accepted* states, giving the community enough time to provide feedback. After which, the *Deprecated*, *Removing*, and *Removed* transition periods will vary by the type and scope of the technical change.
  • Proposed on Day 1
  • Communicated from Day 2 to Day 13
  • Accepted on Day 14 (depending on influx of feedback)
  • Deprecated/Removing/Removed - from Day 15 onwards (depending on resources and technology being removed)

Document

Do the following to document your proposal (see DEPR-6 as an example):

  1. Create a DEPR ticket on the Deprecation and Removal JIRA board.
  2. Include target dates for: Accepted and Removed (in explicit fields on the ticket). See Timeline for considerations.
  3. Include in which Open edX named release the code will be removed.
  4. Include a description with links to what is being removed, in a Removal section.
  5. Include a description with links to what it is being replaced by, in a Replacement section.
  6. If you plan to mark the code for deprecation, explain how in a Deprecation section. See Deprecated for considerations.
  7. If automated migration will be needed, explain your migration plan in a Migration section.
  8. If there is any additional publicly shareable information or data from your earlier analysis, include that on the ticket.

Now you are ready to communicate your proposal.

Communicated

Announce

Announce your proposal to deprecate and remove to the following communication channels.

To the Open edX Discourse

Post a message to Open edX Discourse Deprecation Announcements, using the following template:

Subject: Deprecation/Removal: <Technology Name> <DEPR-Number>

Body:

Hi there,

We plan to deprecate and remove <Short description of the technology>. We are targeting removal after <Target Removal Date>.

Please read https://openedx.atlassian.net/browse/<DEPR-Number> for more information and to post any questions/comments. The proposed deadline for comments before acceptance is <Target Accepted Date>.

Thanks, <Your name>

To openedx.slack.com

Post the following in the #open-edx-proposals and #general openedx slack channels:

Removal of <*Technology Name>:* We plan to deprecate and remove <Short description of the technology>. We are targeting removal after <Target Removal Date>.

Please read https://openedx.atlassian.net/browse/<DEPR-Number> for more information and to post any questions/comments. The proposed deadline for comments before acceptance is <Target Accepted Date>.

Monitor Feedback

Once announcements are made, update the DEPR ticket to the Communicated state.

Allocate time to be responsive to any and all feedback and input on your DEPR ticket. Update the ticket and the proposal, if neccessary, with any information that should be captured from the ongoing feedback. Continue to iterate and do this until the announced target Accepted date.

If during this time, there is a large amount of churn or concern, be open to adjusting the target dates and revisiting the proposal. If community alignment seems difficult, reach out to the edX Architecture team for directional guidance. In some cases, the proposal may need to be Abandoned entirely.

Accepted

Once enough time is allotted for community feedback, all concerns on the DEPR ticket are responded to, and the target Accepted date has passed, update the state of the DEPR ticket to Accepted.

Deprecated

If you decided to mark the code for deprecation during your Analyze or Monitor Feedback phases, invest time in doing so and update the state of the DEPR ticket to Deprecated once that is completed.

Here are some common ways to mark a technology as deprecated:

  • REST API - Specify in the 1st line of the API’s docstring:

    "Deprecated <DEPR-Number>"
    
  • Javascript code - Add a log statement that executes once without being noisy:

    console.log("<Technology name> is deprecated. See <DEPR-Number>.")
    
  • Python code - Add a warnings.warn statement so it executes once without being noisy:

    warnings.warn("<Technology name> is deprecated. See <DEPR-Number>.", DeprecationWarning)
    
  • Feature toggles - Set the “Expiration Date” as described in OEP-17.

  • Github repo - See OEP-14

  • XBlock - For edx.org specifically, see Deprecating xBlock for the edX website.

Removing

When a team begins development work to remove the code, the DEPR ticket’s state should be updated to Removing.

During this phase, remember the following:

  • Implement the proposed and agreed upon migration path.
  • Remove related code from all places, including the frontend, APIs, and the backend, perhaps even in that order.
  • Remove any related documentation on docs.edx.org and elsewhere.
  • Continue to update the ticket with any delays or issues that may arise, being mindful of the target Removed date set on the DEPR ticket.

Removed

When removal is complete:

  1. Add the DEPR ticket to the wiki page for the next Open edX named release to keep track of which removals occurred in which named release. Note: Eventually, this should be included in a .rst file bundled with the codebase.
  2. Update the DEPR ticket’s state to Removed.
  3. Announce the removal in the #open-edx-proposals and #general openedx slack channels.
  4. Optionally, celebrate with your team by banging and breaking a piñata!
_images/pinata.png