LFX Insights v3 Feedback

This topic is for community feedback on the preview of LFX Insights v3 which is available to early adopters and which is known to have issues.

2 Likes

V3- some nifty features but it still fails to provide the core functionality available in V1 that LFN needs to conduct business.

It is election time for the Nephio Project. Nephio like most of our network projects stipulates that only Active Contributors can vote or run in an election:

An Active Contributor is defined as any member of the Nephio community
who has made twenty (20) or more measurable contributions as assessed 
by the TSC during the previous 12-month period, inclusive of code merged, 
code reviews performed, presentations, technical documentation of software
functions, wiki page edits, or issue tracking activities

Nephio’s decision to include “presentations” as one of their criteria is, uh, interesting, but there is no way to instrument that. Ever. It will be up to the community to figure out how they are going to track the number of presentations given. For everything else in that paragraph however, determining who is qualified to run/vote in the election and building an actual list of email addresses to distribute ballots to is a simple leaderboard query using Insights V1. It is 10 minute job tops.

Unfortunately Nephio was onboarded to Insights V3 which doesn’t support Confluence; I cannot get any wiki stats. Even if it did include Confluence, there is a more pervasive issue which I’ve been reporting since V2 was in beta - key leaderboard functionality LFN depends upon to run our communities was not carried forward into either V2 or V3. The newer leaderboard functionality amounts to a massive regression as far as LFN is concerned. I’ve demonstrated the functionality we need to team members on multiple occasions, but the focus always seems to be on new UI or new feature development.

Up until now that hasn’t mattered to us because we’ve just kept using V1 until something gets done about the leaderboard. Now that Nephio is part of LFN , I’m suddenly faced with two separate regressions in V3, that are preventing me from doing my job, and which I must now explain to the TSC.
:expressionless:

-kenny

2 Likes

Thanks for getting Cilium onboarded into LFX insights, there is some really useful data here! Just a couple of comments / questions from my initial impressions

  • It took me a while to realise that the stats at the top of the overview page are based on the selected time period, and they aren’t totals for all time. For example, I initially thought that the GitHub stars number was wrong since it was showing just under 3k, when cilium/cilium’s total is over 16k. After a while I realised that it’s telling me that we’ve added 3k stars this calendar year. I think others are likely to make the same mistake, but perhaps this could be better sign-posted with slight tweaks to some of the titles in those top six boxes on the overview. For example “Stars added”, “Forks created”, “Issues raised”

  • Under the cilium/cilium overview the Contributors box says that Active Contributors have increased by 35%, but under Geographical Distribution at the bottom of the page, it says Total contributors have decreased by 18.18%. Can you clarify please?

  • Since the project is hosted on GitHub I’m wondering why some activities are classed as “Git” on the Activities report?

  • I’m curious about where the “organization” data for each contributor comes from. Do organizations need to get their members to update their GitHub profiles? What happens if someone changes company?

2 Likes

Hello @lizrice,

Thank you for your feedback and questions. Really great points these.
For #1, we anticipate making subtle design updates to ensure those summary charts are more intuitive.

#2, The reason the number of Contributors differs on the Active Contributors chart versus the Geographical Distribution is because not all contributors will have their location set on their Github profiles and thus the Geographical Distribution chart will show lower numbers. This is a great question though, one that will help us make certain changes that atleast conveys that message on why the numbers differ.

#3, Commits data are from Git, that’s the reason you see some activities classified as “Git” on the Activities Report.

#4, We will come back to you on this in the form of a meeting invite/and or a direct response.

1 Like

Hi @Kenny_Paul , Thank you very much for your feedback.
I am sorry for the inconvenience caused until we support other data sources like Confluence and bring in parity with the earlier version of Insights.
Support for Gerrit, Confluence and Jira are anticipated to be the next on our priority list and I will keep you informed as we get there.

1 Like

Hello, I just saw this for the first time from the Maintainer newsletter. I am one of the Jaeger maintainers. I was noticing the scoring was coming across all of our repos, however, many of those are supportive utilities and other contributions. Some of them even have different maintainers. The main work we do is across two repos, yet we have 35 total repositories. We will clean up some of these and likely have a couple of dozen. How can we get the right data and scoring without including everything in our organization? I have joined the feedback channel on Slack if you’d like to speak async.

Hi @jkowall. Excellent, let’s continue the conversation in that public Slack channel.

Hello, Talking with @cra about insights and devstats and he asked me to post here. I have the following questions.

  • Insights vs Devstats. Is there a difference? Are they both getting updated concurrently? Is one more “complete” than the other?
  • On insights, What constitutes an “activity”? Does a doc update count the “same as” a code change? Does commenting on a PR count “the same as” a code PR?
  • ^ Follow up: Are the stats weighted in anyway? (large code PR vs 1-2 lines)
  • How are Organizations rated? Does me submitting a code PR and a co-worker submitting another (unrelated) PR count as “one” for our Org or “two”? (using simple “points” as an example)

My questions are generic, but I’m specifically focused on the Argo Insights and Argo DevStats

1 Like

Hi Christian, thank you so much for the feedback. I’ll be responding with answers by end of day tomorrow. Appreciate it! Kieran

Hello Christian,

Below are some responses to your questions:

  • Insights vs Devstats. Is there a difference? Are they both getting updated concurrently? Is one more “complete” than the other?

  • *Insights is being actively worked on and improved by a team and regularly updated. Insights has more data by including data from non-code contributions and visualizes the project data differently than Devstats.

  • On insights, What constitutes an “activity”? Does a doc update count the “same as” a code change? Does commenting on a PR count “the same as” a code PR?

  • Here is the documentation on what constitutes an activity for Insights - found here under Contributor activities tab Activities Types | Linux Foundation Documentation

  • ^ Follow up: Are the stats weighted in anyway? (large code PR vs 1-2 lines)

  • Currently, no we do not weight the stats. We do visualize lines of code for overall review. We can add considering a weight to the contributions to our backlog for consideration. How would you expect the weighting to assist you as a reviewer of an open source project? Why would that be useful from your perspective?

  • How are Organizations rated? Does me submitting a code PR and a co-worker submitting another (unrelated) PR count as “one” for our Org or “two”? (using simple “points” as an example)

  • We affiliate Organizations by where the Contributor worked when they made the contribution. The example above would currently be counted as 2 for your org as written in the example.

My questions are generic, but I’m specifically focused on the Argo Insights 1 and Argo DevStats 1

1 Like

Thank you Kieran!

That was useful information, as to your question.

How would you expect the weighting to assist you as a reviewer of an open source project? Why would that be useful from your perspective?

I was mainly asking so that I understand the stats. It would be nice to have weighted contributions tracked as it’ll aid when calling for a vote to promote someone from reviewer to merger (for example). Right now, this is done manually (looking at the users contributions individually before a vote). But this isn’t something that’s pressing.