Orchestral strings is good for backup. So is a piano or organ. They are not so much for the melody, trumpets and horns would rather play most of the melody. Brass instruments lead the orchestra and are backed up by strings, woodwinds, and percussion.
Conical-bore and double-reed instruments (such as saxophones and bassoons) make better bass, clarinets (which have single-reed and cylindrical bore) make better alto, and flutes and recorders make better soprano and sopranino. So a quartet of a bassoon, two clarinets, and a piccolo, or a quintet of a bassoon, two clarinets, and two flutes (one doubling piccolo) would be good.
Saxophones make better for a wide range of pitches than clarinets do. Clarinets make more for treble pitches. To be more precise, saxophones with multiple parts (alto, tenor, and baritone) close in pitch, and clarinets with two parts (treble and bass) far apart, double reeds with two or three parts (oboes, bassoons, and maybe contrabassoons) again far apart. Another thing to be more clear, oboes and clarinets are equally important for treble pitch, while conical-bore and double-reed instruments are more important than clarinets in bass and contrabass pitch. Don’t have contrabassoons without a bass clarinets, or bass clarinets without contrabassoons, the latter at least not in orchestras. Probably not have both clarinets and oboes without bassoons.
For low brass, have a saxhorn (such as tuba or baritone) or trombone rather than bass trumpet. For alto brass instrument, have a French horn rather than an alto horn or mellophone. Have tuba and/or bass or contrabass trombone, rather than cimbasso.
A thumb valve is more important for a French horn than the F-trigger is for a tenor trombone. Rotary trigger valves are also more important for bass and contrabass trombones than for tenor trombones. Tenor trombones have only one rotary trigger valve, if any, and bass and contrabass trombones have one or two rotary trigger valves, no more, no less.
Basso continuo is something harmonic and maybe bass (such as a piano, organ, harp, or guitar) that fills an orchestra. Makes an orchestra sound more full. Piano, hit drums, and guitar are hard, bowed strings are soft. Hard (such as percussion and plucked or struck strings) complements soft (such as bowed strings) well, but an organ which is soft (as opposed to hard) nevertheless could fill. Brass and wind also complement the strings.
Piano combines well with a trumpet, cello, and violins, all four go together. Piano is a poor mix with a guitar, unless maybe the guitar is solid-body or semi-hollow-body electric. Piano doesn’t go too poorly with electric bass, especially if other instruments are involved. Guitar also goes well with trumpet and violins. Piano combines well with an orchestra and melds with string basses. Overdriven guitar makes an orchestral hit more grand, powerful, and dramatic.
In a wind instrument or bars (such as xylophone, vibraphone, marimba, or glockenspiel), polymer and wood is better for vibrating thing (such as the reed of a wind instrument or the bars of a xylophone), and metal is better for the resonator. In other words, polymer vibrator and metal resonator is better than metal vibrator and polymer resonator. However, it’s kind of the opposite for string instruments.
If you want subcontrabass, there’s a subcontrabass tuba. Don’t care for an octocontrabass or subcontrabass clarinet, much less a hyperbass or subcontrabass flute. Certainly not an octosubcontrabass or sub-subcontrabass recorder. They don’t sound good, a subcontrabass flute or recorder wouldn’t be heard well due to few harmonics, and the subcontrabass clarinet would sound weird due to the dominance of odd-numbered harmonics, whereas a subcontrabass sax and subcontrabass tuba would sound great due to their rich harmonics. A subcontrabassoon would sound awesome, but it’s not built as far as I know. A contrabassoon can play about as low as a subcontrabass sax.
In an organ, reed pipes make better bass, flue pipes (especially open diapasons and tapered hybrids) make better treble. Edit: Of all flue pipes, stopped wooden cuboidal flutes make a great foundation, especially if they have rectangular cross section but are not good on their own. Chimney flutes make good bass as well, not so much if they have cylindrical chimneys which leads to inharmonic partials, but if they have conical/funnel chimneys as in Gemsrohrflote and Cone Lieblich Gedackt which would be great on their own and with chorus.
For keyboard instrument, have a piano or clavichord rather than harpsichord. For plucked strings, have a harp or guitar rather than a harpsichord. For struck strings, have a piano or clavichord rather than a dulcimer.
Also, clavichord should definitely be electric with magnetic pickup. That solves the problem of limited volume, so that it’s not limited to intimate performance. Also, clavichord is cutting edge so is great in electric form.
Electronics make better bass, acoustics make better treble, but it would be desired to have at least some synth in treble and have a few acoustics (such as string basses, tubas, and bassoons) in bass. Electronics should be more penetrating than acoustics. So replacing saxophones with sawtooth wave is more desired than replacing clarinets with square wave or certainly flutes with sine waves. Saxophone or contrabass trombone wouldn’t due well compare to electronics.
Plucked strings (e.g. guitar) are better electric, bowed strings are better acoustic. So have electric guitar and acoustic bowed strings, not electric bowed strings and acoustic guitar. Bowed strings could however use StringAmp, namely the concertmaster, principal second violin, and principal cello. Piano also makes better acoustic, so better to have electric guitar and acoustic piano rather than have electric piano and acoustic guitar.
In electronic or combined electronic+acoustic music, either have an acoustic or electroacoustic or semi-acoustic piano or have an electronic piano built into keyboard synth workstation rather than have a separate digital piano. Electric tines or reeds piano could work though.
Don’t have electric guitar and drum machine without synthesizer. More important to add synth to rock band than to replace drum kit with drum machine. So have an electric guitar and synthesizer with either drum kit or drum machine. Solution: have an electronic drum kit (combining the rhythmic expression of drum kit with the versatility – imitation of many drum kits – of a drum machine), electric guitar, and synth. Maybe the hi-hat and cymbals still be acoustic, because variability in tone is not as important for these. Also, maybe the kickdrum be done by the drum machine, because steady beat is more important for the kickdrum, and rhythmic expression is more important for the snares and tom-toms, and steady beat is less important for tom-toms, hi-hats, and cymbals. Maybe the snares done by both drum machine and drum kit, because both rhythmic expression and steady beat are important for snare.