Cortex Engineering Intelligence vs. Jellyfish: The Best Engineering Metrics Platforms

With more scrutiny on budgets and bandwidth, engineering leaders need robust tools to measure productivity, identify bottlenecks, and align their efforts with business objectives. Platforms like Jellyfish have emerged, focusing solely on this problem, providing in-depth engineering intelligence and productivity metrics. Alternatively, Cortex—an Internal Developer Portal (IDP)—has started integrating engineering intelligence features into their IDP product.

This article delves into a detailed comparison between Cortex and Jellyfish, examining the metrics they offer, their usefulness, and how they stack up against each other in empowering engineering leadership.

What is Cortex?

Cortex is an Internal Developer Portal with similarities to tools like OpsLevel or Backstage. It aims to streamline the development process by providing a centralized hub for developers to access services, documentation, and tools. Recently, Cortex introduced its Engineering Intelligence features, adding basic productivity metrics to help teams monitor their performance.

What is Jellyfish?

Jellyfish is a platform dedicated exclusively to engineering intelligence and productivity metrics. It offers a comprehensive suite of tools that provide deep insights into engineering activities, resource allocation, and how these align with business goals. Jellyfish specializes in metrics that intersect finance and engineering, enabling leadership to make data-driven decisions that enhance both productivity and profitability.

Cortex's Engineering Intelligence Metrics

Cortex's Eng Intelligence includes the following metrics:

• Average PR Open to Close Time
• Average Time to First Review
• Average Time to Approval
• Average Deploys per Week
• Deploy Change Failure Rate
• Mean Time to Resolve Incidents
• Pull Requests Opened
• Weekly Pull Requests Merged
• Average Pull Requests Reviewed per Week
• Average Commits per Pull Request
• Average Lines of Code Changed per Pull Request
• Incidents Opened
• Incidents Opened per Week

These metrics are designed to provide insights into the development workflow, aiming to help teams improve efficiency and reduce bottlenecks.

At first glance, the metrics provided by Cortex seem valuable for several reasons:

• Baseline Productivity Insights: Metrics like average PR open to close time and average time to first review offer a basic understanding of the team's workflow efficiency.
• Incident Management: Tracking incidents opened and mean time to resolve can help teams respond faster to issues, potentially improving system reliability.
• Deployment Frequency: Monitoring average deploys per week can indicate how quickly new features and fixes reach production.
• Code Review Engagement: Metrics on pull requests reviewed per week and average commits per PR might encourage more collaborative code reviews.

These data points can serve as starting points for teams looking to gain visibility into their development processes.

Drawbacks of Cortex's Engineering Intelligence Metrics

Despite the initial appeal, there are several limitations to Cortex's engineering intelligence metrics that may hinder their effectiveness:

1. Lack of Contextual Understanding

• Surface-Level Data: Metrics such as average PR open to close time do not consider the complexity or size of tasks. A minor bug fix is equated with a major feature, potentially skewing productivity assessments.
• No Differentiation of Work Types: Without distinguishing between new feature development, maintenance, or urgent fixes, the metrics provide an incomplete picture of engineering efforts.

2. Encouragement of Counterproductive Behaviors

• Quantity Over Quality: Emphasizing the number of PRs opened or average commits per PR may incentivize engineers to split work unnecessarily to inflate metrics.
• Speed Over Thoroughness: Focusing on quick approvals and merges might lead to rushed code reviews, compromising code quality and increasing the likelihood of defects.

3. Lack of Actionable Insights

• No Root Cause Analysis: Metrics like mean time to resolve incidents highlight issues but do not help identify underlying causes or suggest solutions.
• Missing Guidance for Improvement: The data lacks actionable recommendations, making it challenging for teams to implement effective changes.

4. Does Not Reflect Team Health and Collaboration

• Overlooks Team Dynamics: There's no measurement of team morale, communication effectiveness, or collaboration, all of which significantly impact productivity and project success.
• Individualistic Metrics: Focusing on individual output can undermine teamwork, as engineers may prioritize personal metrics over team goals.

5. Potential for Misinterpretation and Misuse

• Gaming the System: Awareness of these metrics might lead to behaviors aimed at improving numbers rather than actual productivity or quality, such as increasing commit counts without meaningful contributions.
• False Indicators of Progress: Improved metrics do not necessarily correlate with delivering customer value or achieving strategic business objectives.

6. Insufficient for Strategic Decision-Making

• No Alignment with Business Objectives: The metrics don't tie engineering activities to business outcomes, making it difficult for leadership to assess the impact of engineering efforts on company goals.
• Lack of Financial Insight: Important aspects like resource allocation and the financial implications of engineering work are not addressed.

Advantages of Jellyfish’s Metrics and Approach

Jellyfish addresses many of the shortcomings found in Cortex's engineering intelligence features by providing advanced, actionable metrics that align engineering efforts with business objectives.

Outcome-Oriented Engineering Metrics

Work Allocation Analysis

Jellyfish offers insights into how engineering time is divided among:

• New Feature Development: Understanding the investment in innovation and growth.
• Unplanned Work: Quantifying time spent on unexpected tasks like bug fixes or urgent issues.
• Maintenance and Technical Debt: Assessing efforts to improve long-term codebase health.

Quantifying Unplanned Work

By measuring the amount of unplanned work each team handles, Jellyfish helps identify:

• Process Inefficiencies: High levels of unplanned work may indicate systemic issues that need addressing.
• Impact on Planned Projects: Unplanned tasks can derail project timelines, affecting overall productivity.

Measuring Team Thrash

Jellyfish can assess team thrash, which includes:

• Frequent Context-Switching: High levels of thrash reduce focus and efficiency.
• Priority Changes: Understanding how shifting priorities impact team performance.

Project Schedule Tracking

• Identifying Delays: Highlighting which teams or projects are behind schedule.
• Resource Reallocation: Providing data to make informed decisions about where to allocate resources to meet deadlines.

Financial Impact Assessment

• Cost of Inefficiencies: Translating unplanned work and delays into financial terms.
• ROI Calculations: Helping leadership understand the return on investment for different projects and initiatives.

Actionable Insights with Recommendations

• Root Cause Identification: Analyzing underlying causes of issues, not just the symptoms.
• Strategic Improvement Suggestions: Offering data-driven recommendations for process enhancements or team restructuring.

Alignment with Business Objectives

• Linking Engineering to Business Goals: Ensuring that engineering efforts directly contribute to the company's strategic objectives.
• Value Delivery Measurement: Evaluating how engineering work impacts customer satisfaction, revenue growth, and competitive advantage.

Enhanced Decision-Making Support

• Resource Allocation Insights: Understanding where engineering resources are most effectively utilized.
• Financial Forecasting and Budgeting: Predicting future costs associated with engineering projects for better financial planning.

Comprehensive View of Engineering Health

• Team Dynamics and Collaboration Metrics: Assessing factors like team morale and communication efficiency.
• Risk Identification and Management: Early detection of potential risks in projects or processes.

Conclusion

While Cortex provides a foundational set of engineering metrics within its Internal Developer Portal, these metrics may be too simplistic and lack the depth required for strategic improvement. Jellyfish offers a more sophisticated, integrated approach that delivers valuable insights at the intersection of engineering and the business.

However, to maximize the benefits of engineering intelligence derived from Jellyfish, integrating it within a dynamic Internal Developer Portal like OpsLevel offers the best of both worlds. OpsLevel not only streamlines development processes by providing a centralized platform for services, documentation, and tools but also offers advanced features that enhance team productivity and collaboration.

By combining Jellyfish's in-depth analytics with OpsLevel's robust IDP capabilities, organizations can achieve:

• Comprehensive Visibility: Gain a holistic view of engineering activities, from high-level metrics to detailed project workflows.
• Aligned Objectives: Ensure that engineering efforts are tightly aligned with business goals, facilitated by OpsLevel's service ownership and maturity features.
• Enhanced Collaboration: Leverage OpsLevel's tools to improve team communication and coordination, amplifying the impact of Jellyfish's findings.
• Streamlined Processes: Use OpsLevel's automation and self-service capabilities to address issues identified by Jellyfish efficiently.

In the comparison between Cortex and Jellyfish, Jellyfish stands out for its advanced, actionable metrics. But when integrated with a dynamic IDP like OpsLevel, organizations unlock the full potential of their engineering teams, driving greater value for the company and its customers.

For more information on how integrating advanced engineering intelligence platforms with a dynamic Internal Developer Portal can benefit your organization, book a call with the OpsLevel IDP team or explore our resources on optimizing engineering productivity.