Hệ phương trình \(\left\{ \begin{array}{l}{x^3}y\left( {1 + y} \right) + {x^2}{y^2}\left( {2 + y} \right) + x{y^3} - 30 = 0\\{x^2}y + x\left( {1 + y + {y^2}} \right) + y - 11 = 0\end{array} \right.\) có bao nhiêu cặp nghiệm \(\left( {x;y} \right)\)  mà \(x < 1\) ?

Hệ tương đương với : \(\left\{ \begin{array}{l}xy\left( {x + y} \right)\left( {x + y + xy} \right) = 30\\xy\left( {x + y} \right) + x + y + xy = 11\end{array} \right.\).

Đặt \(xy\left( {x + y} \right) = a;xy + x + y = b\). Ta thu được hệ:

\(\left\{ \begin{array}{l}ab = 30\\a + b = 11\end{array} \right. \Leftrightarrow \left[ \begin{array}{l}a = 5;b = 6\\a = 6;b = 5\end{array} \right. \Leftrightarrow \left[ \begin{array}{l}\left\{ \begin{array}{l}xy\left( {x + y} \right) = 5\\xy + x + y = 6\end{array} \right.\\\left\{ \begin{array}{l}xy\left( {x + y} \right) = 6\\xy + x + y = 5\end{array} \right.\end{array} \right.\).

TH1: \(\left\{ \begin{array}{l}xy\left( {x + y} \right) = 6\\xy + x + y = 5\end{array} \right. \Leftrightarrow \left[ \begin{array}{l}\left\{ \begin{array}{l}xy = 2\\x + y = 3\end{array} \right.\\\left\{ \begin{array}{l}xy = 3\\x + y = 2\end{array} \right.(L)\end{array} \right. \Leftrightarrow \left[ \begin{array}{l}x = 2;y = 1\\x = 1;y = 2\end{array} \right.\)

TH2: \(\left\{ \begin{array}{l}xy\left( {x + y} \right) = 5\\xy + x + y = 6\end{array} \right. \Leftrightarrow \left[ \begin{array}{l}\left\{ \begin{array}{l}xy = 5\\x + y = 1\end{array} \right.(L)\\\left\{ \begin{array}{l}xy = 1\\x + y = 5\end{array} \right.\end{array} \right. \Leftrightarrow \left[ \begin{array}{l}x = \dfrac{{5 - \sqrt {21} }}{2};y = \dfrac{{5 + \sqrt {21} }}{2}\\x = \dfrac{{5 + \sqrt {21} }}{2};y = \dfrac{{5 - \sqrt {21} }}{2}\end{array} \right.\).

Vậy hệ có nghiệm: \(\left( {x;y} \right) = \left( {1;2} \right),\left( {2;1} \right),\left( {\dfrac{{5 \pm \sqrt {21} }}{2};\dfrac{{5 \mp \sqrt {21} }}{2}} \right)\)

Suy ra có một cặp nghiệm thỏa mãn đề bài là \(\left( {\dfrac{{5 - \sqrt {21} }}{2};\dfrac{{5 + \sqrt {21} }}{2}} \right)\) .