Re: [cooker-amd64] Re: using and compiling rpms not specifically put together for amd64

[Stefan van der Eijk]

Duncan wrote:

> Arie Folger posted <200401231339.33560.afolger@aishdas.org>, excerpted
> below,  on Sat, 24 Jan 2004 19:11:59 +0100:
>
>  
>
> > Can one assume that by compiling SRPMs the result will automatically be a
> > program linked to the 64 bit libraries whenever available, does one need a
> > special config flag, or does it require that the source code be adapted?
> >
> > Also, there are considerably more contrib packages for the i586 than the
> > amd64 (IIRC there are none yet for the amd64). Those packages have
> > dependencies that aren't resolvable using the amd64 libraries only. Can
> > one simply install a 32 bit version alongside the 64 bit version of a
> > library, and simply install the i586 package?
> >    
>
> Oh... boy..  Someone else embarking upon the learning curve I started on
> myself 2-3 months ago.  <g>  I knew, getting into amd64 this early, that
> I'd be learning a lot, but expected thru all the frustration, the learning
> experience would be fun, and I'd come out the end knowing more about my
> new hardware platform of choice, Mandrake and RPM, and Linux as a software
> platform, as well as computers in general, and consider it all well worth
> it.  How about you? <g>  (If it sounds like I'm saying you have a big job
> ahead of you, it's because you do, altho with the full 9.2 for AMD64
> release out now, it should be a bit easier and less frustrating  now, tho
> without some of the "learning opportunities" <g> as well.)
>
> A single sentence summary would be that the situation is at once simpler
> and more complex than you suggest in your question.  However, again,
> that's a good opportunity to learn, right?
>
> First, dealing with contribs.  Yes, there ARE contribs out for amd64, now,
> altho last I checked they weren't urpmified/hdlistified yet, so you have
> to check and download them manually instead of using urpmi to do it.
>  
You're right. Please ask warly to fix this. I don't recall if I've asked 
him to do this, but there are a number of other things I have requested 
and am still waiting for. Most important, IMHO is to fix a bug in rpmctl 
which will let it understand what "amd64.rpm" and "sparc.rpm" packages are.

I'm going to tell you: Mandrake doesn't have a lot of resources. And 
they haven't (yet) got everything in place to deal with external 
contributors and to let them scale development over them. This is a 
rocky road, sometimes with successes, but mostly getting things done 
means bugging (nagging) MDK employees and begging them to do things. OK, 
you're warned now.

> Stephan van der Eijk has put together an automated build script that takes
> the existing cooker, contribs, and plf srpms, and rebuilds as it
> can the amd64 (and other platforms like sparc) binary packages.  
>
> Sometimes these packages rebuild without issue, in which case they are
> uploaded to the mirrors (of which there aren't yet so many for the amd64
> packages, especially contribs, but..), sometimes there are problems, in
> which case the bot logs the errors and Stephan can notify the package
> maintainers.  (I'm not sure if he has a bot automatically spamming the
> maintainers about their problems or if he does that manually, but..)
>  
I've had a spamming bot running a few times. My experience is that the 
issues that could get solved don't get solved and that people focus on 
trying to prove that SlBd isn't doing a good job / SlBd is broken, etc. 
Of course SlBd isn't perfect, but instead of blaming the bot I'd expect 
people to at least fix the issues that are valid.

I could write an interface to file bugs in bugzilla, but to do this 
properly and to maintain it, we'll need some more rights in bugzilla.

> These problems can range from the package being exclusive-arch or
> binary-only, examples being binary-only audio/visual codecs and i586
> exclusive emulation packages, to rpm spec files that haven't been adjusted
> yet to take bi-arch architectures like amd64 into account.
>  
Ask Gwenole :-) He spent > 3 weeks getting tonnes of packages mklibbed. 
Even packages that he doesn't maintain --> that the maintainers should 
be fixing. It's very frustrating to see collegues not doing their job 
and having to mop up after them.

A lot of packages still need to be fixed.
- A lot of packages use "32bit" BuildRequires --> these BuildRequires do 
not exists on a amd64 / sparc64 system and should be changed to a 
"generic" BuildRequires;
- A lot of packages need to be mklibbed;
- Quite some packages have scripts in /usr/lib/ that will end up in 
/usr/lib64/ on amd64 / sparc64. Autofs was one of them.

and then there's testing and reporting bugs. Issue with this is that 
contributors can report a bug but then have limitted capabilities in 
bugzilla to do other things with them. Confirming a bug requires special 
access.

a lot of bugs don't get closed and clutter up bugzilla, reducing it's 
effectiveness.

> Here's his site, with the various logs and status reports from his
> build-bot.  You may or may not find it useful.  I certainly do at times,
> as it allows me to confirm that I'm not the ONLY one having trouble
> rebuilding a particular RPM.
>
> http://eijk.homelinux.org/build/amd64/
>
> Up under build, you can find the script he uses, and thus, figure out a
> bit better what causes the build-logs to be placed in one subdir or
> another, if desired.
>
> This is apparently the primary mirror he uploads to, when the build
> succeeds, so should have the newest available contribs for amd64, anyway. 
> (Main gets built by Mdk, elsewhere, tho it's mirrored there also.)
>  
Please verify is main is being built by mdk. :-)

====
$ rpm -qpi 
/mirrors/Mandrake-devel/cooker/amd64/Mandrake/RPMS/mutt-utf8-1.5.5.1i-2mdk.amd64.rpm 

Name        : mutt-utf8                     Relocations: (not relocateable)
Version     : 1.5.5.1i                      Vendor: MandrakeSoft
Release     : 2mdk                          Build Date: Sun 25 Jan 2004 
10:32:42 GMT
Install date: (not installed)               *Build Host: 
turaco.mandrake.vpn*
Group       : Networking/Mail               Source RPM: 
mutt-1.5.5.1i-2mdk.src.rpm
Size        : 703112                        License: GPL
URL         : http://www.mutt.org/
Summary     : Text mode mail user agent supporting wide character
Description :
Mutt is a text mode mail user agent. Mutt supports color, threading,
arbitrary key remapping, and a lot of customization.

You should install mutt if you've used mutt in the past and you prefer
it, or if you're new to mail programs and you haven't decided which
one you're going to use.

NOTE: This version of mutt is linked against ncurses with wide char
     support, and is useful for, say, people using UTF-8 locales.
====

So, that's in the open too...

> ftp://anorien.csc.warwick.ac.uk/
>
> On to your library question, the answer to which explains some of the
> problems the rebuilds have as well. Mandrake policy is to keep 64-bit
> libraries in their own lib64 path, exactly to allow one to do as you were
> suggesting above, install both 32- and 64-bit copies of the same
> libraries, without them conflicting, as needed. One of the SRPM rebuild
> problems often encountered is that the SRPM hasn't yet been adjusted to
> take that into account.
i.e. doesn't use mklib.

> Rather than using rpm macros to specify library
> locations, etc, it has them hardcoded to the standard lib locations, and
> the lib64 located libraries it can't find.  Gradually, this is changing as
> the rebuild process continues, and the various SRPMs are macroified to be
> able to build on all platforms without issue.
>
> That leads into another question you asked.  When building for the
> platform you are on, there are no special flags needed, ideally, as the
> RPM macro system should take care of everything, automatically adjusting
> the target architecture and platform as necessary.  Again, however, that
> presupposes that all the macros are set up correctly to do so, and that
> isn't yet the case with some SRPMs.
>  
When you have an up2date rpm & rpm-build on your host, the macros are 
there. The only thing that needs to happen is have the src.rpm adjusted 
to use them.

> Other times, the problem building the SRPM is that build-dependencies it
> needs aren't yet able to build, due to their own problems. Or, they can
> build, but they can't install, because they depend on other packages that
> won't build to run. (Sometimes, this can just be a wrong dependency in
> the RPM. This is often the case with the various perl script packages and
> others that depend on them, such as urpmi, rpmdrake, and the various
> other *drake* packages, for example, since perl dependency checking is an
> as yet imperfect art in itself, not entirely able to be reliably done,
> automatically.  Again, the build-bots mark the error, which then needs
> check, and wrong dependencies removed, or corrected as necessary, manually.)
>
> In particular, there's a growing list of *drake* cooker packages now
> stalled on perl-GTK2, which won't build, for various reasons, in addition
> to a couple bad perl dependencies that may be corrected by now, I'm not
> sure, as I have the packages in my urpmi skip-list, since they DO have an
> unmet valid dependency on perl-GTK2, of a version that won't build yet for
> amd64.  I also have a second stoppage, in that I can't install the latest
> sash, because it depends on the latest glibc package, which won't yet
> build for amd64, due to some sort of problem with i18n, it would appear,
> from Stephan's logs.
>
> Both of those have been blocked for something like two weeks, now, I
> think. There have been continuing *drake* package updates in the mean
> time, so I'm now several versions behind, but there's nothing I can do
> about it since I'm not a coder or packager yet,
Please get an account at Mandrakesoft. Contact Lenny or Warly to get 
one. We need your help.

> and the problems are to
> deep for me to figure out and work around on my own.  That's where the
> frustration sets in, but at least I know they know about it, and are
> working on it, as Stephan's bot logs show I'm not the only one that
> couldn't build the problem packages.
>  
:-/

Filed a bugreport yet at qa.mandrakesoft.com?

> The ones that are the real bugger, however, to get working, are the ones,
> like the core of the glibc and perl-GTK2 package problems above, that
> aren't rpm macro or simple dependency related, and aren't simple rejects
> because they're binary-only or exclusive arched packages that won't run in
> 64 bit amd64 mode. They have deeper issues, like 64-bit unclean code, or
> other assumptions that prove wrong on a platform other than i586.  Code
> that isn't 64-bit clean generally comes down to an assumption the original
> coder made about the length of 32-bit ints (integers) being the same size
> as memory pointers, with the problem being that pointers, on 64-bit apps,
> are 64-bits long, while ints are normally still only 32-bit.  There are
> other similar cases, but that's by far the most common, from what I've
> read.  Obviously, the fix here is to correct the source code, returning
> the patch to the application maintainers, if it's not Mandrake, hoping for
> inclusion, so Mdk doesn't constantly have to "re-fix" each new version. 
> Most application maintainers (aka lead developers) are happy for the help
> and include the fix right away. Others may not. In any case, tho, that's
> far beyond something a bit of srpm package tweaking is going to fix, thus,
> far beyond anything I can work around on my own, so I'm dependent on
> others to work the fix into the pipeline for me.
>
> Well, that's about it for me, except for some lose ends to wrap up.
>
> 1) The contrib packages at the above location should help a lot.
>
> 2) If you haven't done so yet, find and configure an updates mirror for
> use with urpmi, as there are some significant updates out there for a
> number of packages, both security and functional updates.  In particular,
> note that the latest AMD64 kernel packages are 2.4.22-27.

Not completely true... there are some 2.6.1 kernels on the filesystem at 
anorien.csc.warwick.ac.uk:

/build/amd64/contrib/RPMS/kernel-tmb-2.6.1-1.tmb.1mdk-1-1mdk.amd64.rpm
/build/amd64/contrib/RPMS/kernel-tmb-2.6.1-1.tmb.2mdk-1-1mdk.amd64.rpm
/build/amd64/contrib/RPMS/kernel-tmb-2.6.1-1.tmb.3mdk-1-1mdk.amd64.rpm
/build/amd64/contrib/RPMS/kernel-tmb-2.6.1-1.tmb.4mdk-1-1mdk.amd64.rpm
/build/amd64/contrib/RPMS/kernel-tmb-2.6.1-1.tmb.5mdk-1-1mdk.amd64.rpm
/build/amd64/contrib/RPMS/kernel-tmb-2.6.1-1.tmb.6mdk-1-1mdk.amd64.rpm

But these aren't available through http / ftp access :-/

> Not only do
> these close a couple security holes, but release 27 has proven FAR more
> stable on my system than release 24 was, and I'm not crashing as much.
>
> 3) Unfortunately, the flip side of the coin of the build-bot doing the
> stuff that can be done automatically, and Stephan's notifying maintainers
> about problems, is that it's the easy problems that have been fixed, with
> the real difficult problems (and packages depending on difficult case
> problem packages) being the majority of the ones that remain. Thus, if an
> amd64 package isn't already out there for it, as time goes on, it's
> meaning the ones that are left are becoming harder and harder for one to
> simply up and rebuild one's self, because their the ones that no one else
> has been able to fix, just yet.
>  
:-/

> 4) KDE packages deserve special mention, because of their dependency not
> only on KDE libs, but on QT libs as well.  The QT3 libs are a special
> case, due to their rather strange path on bi-arch systems.  All the qt3
> stuff is under /usr/lib/qt3 (NOT /usr/lib64/qt3)  That includes both the
> headers, necessary for rebuilding, and the library binaries, under their
> own lib64 subdir, thus, /usr/lib/qt3/lib64.  Packages that haven't already
> been fixed are thus a bit more difficult than most to get working on your
> own.  Not only must you go in and tweak the  various --with switches in
> the spec or make file manually, but often times in doing so, you find it
> wants to either make that path /usr/lib64/qt3/lib64, or /usr/lib/qt3/lib,
> changing either both or neither to the 64, instead of changing only the
> latter one.  The easiest way to fix this purely for my own builds, since I
> wasn't worried about and had no intention of installing 32-bit qt libs,
> was to use two appropriately place symlinks.  the first being
> /usr/lib64/qt3, pointing to the qt3 dir under /usr/lib, and a second,
> /usr/lib/qt3/lib, pointing to /usr/lib/qt3/lib64.  If I ever accidentally
> install 32-bit qt libs, I'll have one **** of a mess trying to clean up
> the 32-bit libs that overwrote the 64-bit ones due to the symlinks I
> created, but it DID provide a temporary fix for the problem of the srpms
> either adjusting both or neither of the libs, when they SHOULD have been
> only adjusting the latter one, in the absence of my knowing how to
> CORRECTLY fix the rpm macros in question.  Again, this isn't the PROPER
> fix for the problem, but in the absence of knowing enough about rpm macros
> to figure out the PROPER fix, this was an adequate workaround for me only,
> since I wasn't making the resulting packages public.
>  

The nice thing about the kde packages are Mandrakesoft is that the KDE 
team have locked down the upload process so *THEY* can only upload kde 
packages.

Even if you *do* find a way to sneak one by them, you'll get your arse 
toasted in the mailinglist. Blaming you for everyhting that ever went 
wrong even though the *technical* truth is different. But that doesn't 
matter for them, even though the package was already broken, you 
uploaded it a 2nd time, and get blamed for everything.

You can try to communicate with them via private e-mail, but they'll 
ignore you. Bugreports via bugzilla won't get confirmed --> they'll 
ignore you. Putting preasure on them via e-mail (and CC: cooker) seems 
to be the only way to do it, but you won't end up with a friendly 
relationship with them either. I've looked for a win-win situation, but 
there doesn't seem to be one. Contributors are the plebs, and are best 
to be ignored. Ports don't matter, if stuff on i586 doesn't build (in 
slbd) doesn't matter either. "it builds here" is enough.

Conclusion: just forget KDE, for now.

> 5) KDE packages again:  Note that the KDE packages now in i586
> cooker are newer versions than what we have back here on amd64 cooker, 
> KDE 3.2, rather than the KDE 3.1.3+ packages we have.  At one point, I got
> VERY frustrated attempting to compile a KDE package, thinking the problem
> was the libs path issue above, because the error looked the same
> (something about a QT3-mt=multithread capable library being unfound IIRC),
> when it was REALLY that I was attempting to build the new package, with
> its dependencies on the new library, on my system with the OLD library.  I
> spent several hours trying to figure out what was going wrong, and
> swearing at the thing, before I figured out my mistake, and very
> sheepishly went and downloaded the CORRECT SRPM! <g>
>
> 6)  All some SRPM rebuilds need (or did need, when I rebuilt them, they
> are probably already available in binary package form now) is the --target
> x86_64 switch handed to rpm for the rebuild.  These packages were upgraded
> to work with amd64 for the Mdk Corporate-server for AMD64 projects, before
> they decided to call the arch amd64, for 9.2.  Now, the two are basic
> synonyms, with the automatic target amd64 that rpm attempts to apply from
> the site based platform settings set to call x86_64 where necessary, but
> some of the earlier packages don't recognize the amd64 form, and need
> x86_64 specified as the target specifically.  Then they compile fine. 
> Others, such as the KDE packages mentioned above, may need BOTH the
> x86_64 target specified, AND some additional tweaking, such as the
> symlinks I installed, as mentioned above.  Again, I expect most of these
> will already be fixed, however, with binary amd64 rpms available, so you
> may not run into a lot of remaining srpms with that problem.
>
> 7) A warning on libraries vs application binaries!  Note that while
> Mandrake HAS separated the libraries out, so both 32-bit and 64 bit can
> normally be installed and running in parallel, that does **NOT** apply to
> application binaries, or the /bin dirs they inhabit, themselves.  Thus, be
> careful that you don't overwrite 64-bit binary applications, with their
> 32-bit equivalents, while you are upgrading, or whatever.  A typical
> scenario where this might happen would be if you loaded a new urpmi
> source, and it found the 32-bit hdlist instead of the 64-bit one (which
> may not exist yet).  That happened to me at one point with plf.  It wanted
> to upgrade many of my amd64 packages to i586 packages, including
> mplayer and other application packages, which would have meant my current
> 64-bit binaries would have been overwritten by "upgraded" i586 32-bit
> binaries.  **NOT** what I had in mind **AT** **ALL**!  Fortunately, I
> noticed all those i586 tags where I expected to see amd64s, and canceled
> the upgrade.  However, had I just gone ahead and hit enter, I would have
> likely been cursing myself for a day or more, trying to straighten out the
> mess I'd created!
>
> Hopefully, all this info is of help, and doesn't discourage you, but
> rather allows you to pick up the game quickly, and be of help to me, one
> of these days!  <g>  Just keep in mind that it's all fun, and we are
> learning a lot as we go, and a year or two from now, when everyone else
> finally decides 32-bit is going the way of 16-bit and 8-bit before it,
> we'll be old hands at the game.  <g>

duncan, would you like to have an account on the CSC environment, where 
you can build & test amd64 packages? Send over prefered username & your 
ssh public key in a private e-mail to me.

regards,

Stefan

Attachment: smime.p7s
Description: S/MIME Cryptographic Signature