Notice that by the Law of Sines, a:b:c=sinA:sinB:sinC, so let's flip all the cosines using sin2x+cos2x=1 ( sinx is positive for 0∘<x<180∘, so we're good there).
sinA=16315,sinB=815, and \quad\sin C=\dfrac{\sqrt{15}}
These are in the ratio 3:2:4, so our minimal triangle has side lengths 2,3 , and 4 . (A)9 is our answer.
OR
∠ACB is obtuse since its cosine is negative, so we let the foot of the altitude from C to AB be H. Let AH=11x,AC=16x,BH=7y, and BC=8y. By the Pythagorean Theorem, CH=256x2−121x2=3x15 and CH=64y2−49y2=y15. Thus, y=3x. The sides of the triangle are then 16x,11x+7(3x)=32x, and 24x, so for some integers a,b,16x=a and 24x=b, where a and b are minimal. Hence, 16a=24b, or 3a=2b. Thus the smallest possible positive integers a and b that satisfy this are a=2 and b=3, so x=81. The sides of the triangle are 2,3 , and 4 , so (A)9 is our answer.