[Pkg-ofed-devel] New sources for RDMA upstream

[Doug Ledford]

On 9/14/2016 7:37 PM, Jason Gunthorpe wrote:
> On Thu, Sep 15, 2016 at 12:06:34AM +0200, Ana Guerrero Lopez wrote:
>> While the Debian OFED team is mainly me,

Just out of curiosity, if you are packaging up upstream, why do you call
your group the OFED packaging group?  OFED is a specific group of
packages, and not the upstream ones.

>> there are other people
>> contributing from time to time and I'd like encourage them to reply
>> if they have something to add and do not take this reply as any kind
>> of "Debian reply", it's my personal view.
> 
> Hi Ana, yes, I am familiar with the Debian process..
> 
>> I'm well aware of your former involvement with the Debian project,
>> you'll already know of some of my points below, allow me to list them
>> for readers who are not familiar with Debian.
> 
> Indeed, however, I have not CC'd your list and ours together as your list
> is closed.
> 
>> While I understand the rationale from upstream to make a single source
>> repository and release it all at the same time, it doesn't really make
>> Debian's packager life easier. I'd continue packaging things individually
>> and that's likely to mean more work since I'd have to split the code.

I'm not really up on debian guidelines, but is this a requirement or a
choice on your part?

> To be clear, in my view, 're-splitting' (particularly going forward)
> would be a monstrous effort, I can't imagine anyone wanting to do
> that.
>
> It is not my intention that the trees would ever be re-split, and the
> design of the source tree reflects that.
> 
> I recognize that a combined source tree is a *different* kind of work
> from a ditro perspective, and I know that some aspects will be harder
> while others will be easier. For instance the expected introduction of
> the ~4 new providers in the 4.9/4.10 time frame could flow without any
> additional maintainer effort at all vs creating another 4
> ITPs/packages/etc
> 
> You have done a good job identifying the 'harder', but I'd also
> encourage you to think about the positive benefits that working with
> a more active upstream could bring.
> 
>> Bundling it all together makes the packaging quite more complex -as your
>> packaging shows- and the day to day maintenance is also more burdensome.
>> This will raise the Debian contribution bar even more.

One moderately complex package vs. 20 simple packages.  There is
slightly more required packaging acumen required to handle the package
itself, but this is also offset by a reduction in complexity of the
package dependency chain, so building becomes considerably simpler.
There's no real way to quantify which is worse as it depends on what a
packager is more familiar with, so this could be a net loss or win
depending on the person in question.

> Well, I thought the result was fairly reasonable, particularly
> compared to some of the monster packages like gcc, systemd, kde, etc.
> 
> dh automates pretty much everything, it if wasn't for the aesthetic
> desire to have different version numbers for the libraries it would be
> a resonably short rules file.
> 
> The '--fail-missing' behavior of dh_install makes it simple to monitor
> the packaging vs upstream.
> 
>> With the big tarballs every time there is a release, it will take a lot
>> of more of time reviewing all the code, possible copyright changes,
>> packaging updates...
> 
> I'm not sure I understand this. The number of patches will not change,
> they will simply be in one place, not across > 18 different repos.

Indeed.  Code review should be identical in total amount, just different
in that it happens on 1 package instead of 20.  And on any given update,
the only thing that needs to be updated are the changes from the last
tarball.  So if we release a new tarball that only has a one line bugfix
to the built in cxgb4 provider, then that's the only thing that has to
be reviewed.  If you aren't watching the git repos to know what the
changes are, then unpacking both tarballs and doing a recursive diff
between the trees is still better than doing a full review.

> Hopefully the copyright situation will become simpler.. But that is
> always a complex subject :(
> 
>> Every time there is a soname change, it'll delay the availability of
>> all the packages, since the package suffers a rename and it has to
>> be
> 
> We do not intend to ever change sonames.
> 
>> processed manually, the same goes every time there is a new
>> component.
> 
> I'm not certain how many new top level components we will see. Since
> the providers are intended to be bundled into a single package the #1
> source of new packages is eliminated.
> 
> You are are right, that a new component causes work. However, if it
> turns out to be too difficult a distro could choose to not package
> some of the build output - eg you could today choose to defer
> packaging iwpmd.
> 
> As a user, I really appreciate that all relavent components will end
> up being packaged eventually, and that the RDMA support across the
> distros will be more uniform.
> 
>> If there is a building problem, it also delays the full stack.
> 
> Yes, but that is counterbalanced by hopefully a more active upstream
> looking at building issues and keeping track of toolchain progress.

It's also counterbalanced by the argument that this is a good thing.
The RDMA stack is fairly tightly coupled.  If you have a build problem
(which really should be highly unlikely anyway because we will be build
testing before we release a new tarball), it indicates a problem that
should be addressed and so this is an effective safety measure that
helps to prevent a partial update that leaves things in an unknown state.

> As it is today, it appears, you can expect many of the existing
> packages to eventually stop building in unstable because nobody is
> keeping the auto* stuff up to date, or tracking what the gcc community
> is doing.
> 
>> I am also not a fan of CMake, I used to work in the Qt/KDE maintenance in
>> Debian where is used.
> 
> Fortunately dh takes care of pretty much everything related to
> CMake, nobody is asking distributors to touch cmakefiles, and I have
> already done the difficult work of ensuring all the build & install
> works correctly on the main distributions.
> 
>>> I invite you to participate in the upstream community and ensure that
>>> Debian is well represented. If you wish to take over the debian/
>>> directory upstream and maintain the core packaging there (as Roland
>>> did), I'm sure that would be welcomed by the team.
>>
>>
>> I have had a few contributions in linux-rdma in the past :)
>> I really can't compete with people who is paid to do this, as Roland
>> did.
> 
> Less a competition, more a collaboration really..
> 
>> My maintenance of OFED in Debian is done partly on my free time and partly in
>> my daily work time where I'm not paid to do this, but given we use
>> Mellanox OFED, we can re-use some things. I'm hoping for the next release
>> of Debian to have a decent OFED support and also I would like to introduce a
>> few new packages (hfi1 at least is on my list) but I might not make it.
>> I revamped the team a couple of years ago, with the hope that eventually
>> others interested would have joined. It's not happening and I might give up
>> soon.
> 
> I did sense that the OFED group in Debian was understaffed, which is
> why I personally put in the effort to jump start the process with
> packaging so that this significant upstream change did not de-rail
> maintainership in Debian.

If you don't re-split things, I would think this would actually ease the
maintenance requirements.  I know it will for our package people in
Fedora and Red Hat (speaking as the main person who used to do that work).

> I am saddened to hear you thinking about giving up, but I understand
> well how much work volunteering with Debian can be.
> 
> Jason
> 


-- 
Doug Ledford <dledford at redhat.com>
    GPG Key ID: 0E572FDD

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 884 bytes
Desc: OpenPGP digital signature
URL: <http://lists.alioth.debian.org/pipermail/pkg-ofed-devel/attachments/20160914/12c8f318/attachment.sig>