x =8 y = 8

Let \(A=\left[ \begin{matrix} -1 & 1 & 2 \\ 1 & 2 & 3 \\ 3 & 1 & 1 \end{matrix} \right] \)
A = AI
\(\therefore \left[ \begin{matrix} -1 & 1 & 2 \\ 1 & 2 & 3 \\ 3 & 1 & 1 \end{matrix} \right] =A\left[ \begin{matrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{matrix} \right] \)
\({ C }_{ 1 }\leftrightarrow { C }_{ 2 }\)
\(\left[ \begin{matrix} 1 & -1 & 2 \\ 2 & 1 & 3 \\ 1 & 3 & 1 \end{matrix} \right] =A\left[ \begin{matrix} 0 & 1 & 0 \\1 & 0 & 0 \\ 0 & 0 & 1 \end{matrix} \right] \)
\({ C }_{ 2 }\rightarrow { C }_{ 2 }+{ C }_{ 1 }\)
\({ C }_{ 3 }\rightarrow { C }_{ 3 }-{ 2C }_{ 1 }\)
\(\left[ \begin{matrix} 1 & 0 & 0 \\ 2 & 3 & -1 \\ 1 & 4 & -1 \end{matrix} \right] =A\left[ \begin{matrix} 0 & 1 & 0 \\ 1 & 1 & -2 \\ 0 & 0 & 1 \end{matrix} \right] \)
\({ C }_{ 1 }\rightarrow { C }_{ 1 }+{ 2C }_{ 3 }\)
\({ C }_{ 2}\rightarrow { C }_{ 2 }+{ 2C }_{ 3 }\)
\(\left[ \begin{matrix} 1 & 0 & 0 \\ 0 & 1 & -1 \\ -1 & 2 & -1 \end{matrix} \right] =A\left[ \begin{matrix} 0 & 1 & 0 \\ -3 & -3 & -2 \\ 2 & 2 & 1 \end{matrix} \right] \)
\({ C }_{ 3}\rightarrow { C }_{ 3 }+{ C }_{ 2 }\)
\(\left[ \begin{matrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ -1 & 2 & 1 \end{matrix} \right] =A\left[ \begin{matrix} 0 & 1 & 1 \\ -3 & -3 & -5 \\ 2 & 2 & 3 \end{matrix} \right] \)
\({ C }_{ 1 }\rightarrow { C }_{ 1 }+{ C }_{ 3 }\)
\({ C }_{ 2 }\rightarrow { C }_{ 2 }-{ 2C }_{ 3 }\)
\(\left[ \begin{matrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{matrix} \right] =A\left[ \begin{matrix} 1 & -1 & 1 \\ -8 & 7 & -5 \\ 5 & -4 & 3 \end{matrix} \right] \)
\(\Rightarrow { A }^{ -1 }=\left[ \begin{matrix} 1 & -1 & 1 \\ -8 & 7 & -5 \\ 5 & -4 & 3 \end{matrix} \right] \)

For getting A² = \(\left( \begin{matrix} 5 & 0 & 8 \\ 2 & 4 & 5 \\ 8 & 0 & 13 \end{matrix} \right) \)
For getting A³ = \(\left( \begin{matrix} 21 & 0 & 34 \\ 12 & 8 & 23 \\ 34 & 0 & 55 \end{matrix} \right) \)
Simplifying A³ - 6A² + 7A + kI³ as
\(\left( \begin{matrix} k-2 & 0 & 0 \\ 0 & k-2 & 0 \\ 0 & 0 & k-2 \end{matrix} \right) \)
Equating  \(\left( \begin{matrix} k-2 & 0 & 0 \\ 0 & k-2 & 0 \\ 0 & 0 & k-2 \end{matrix} \right) \)
\(=\left( \begin{matrix} 0 & 0 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{matrix} \right) \)
\(\Rightarrow\)  k - 2 = 0
\(\Rightarrow\)  k = 2
Alternative Method :
\(A=\left[ \begin{matrix} 1 & 0 & 2 \\ 0 & 2 & 1 \\ 2 & 0 & 3 \end{matrix} \right] \)
\({ A }^{ 2 }=\left[ \begin{matrix} 1 & 0 & 2 \\ 0 & 2 & 1 \\ 2 & 0 & 3 \end{matrix} \right] \times \left[ \begin{matrix} 1 & 0 & 2 \\ 0 & 2 & 1 \\ 2 & 0 & 3 \end{matrix} \right] \)
\(=\left[ \begin{matrix} 1+0+4 & 0+0+0 & 2+0+6 \\ 0+0+2 & 0+4+0 & 0+2+3 \\ 2+0+6 & 0+0+0 & 4+0+9 \end{matrix} \right] \)
\(=\left[ \begin{matrix} 5 & 0 & 8 \\ 2 & 4 & 5 \\ 8 & 0 & 13 \end{matrix} \right] \)
A³ = A² . A = \(\left[ \begin{matrix} 5 & 0 & 8 \\ 2 & 4 & 5 \\ 8 & 0 & 13 \end{matrix} \right] \left[ \begin{matrix} 1 & 0 & 2 \\ 0 & 2 & 1 \\ 2 & 0 & 3 \end{matrix} \right] \)
\(=\left[ \begin{matrix} 21 & 0 & 34 \\ 12 & 8 & 23 \\ 34 & 0 & 55 \end{matrix} \right] \)
Now A³ - 6A² + 7A + kP₃ = 0
\(\Rightarrow \left[ \begin{matrix} 21 & 0 & 34 \\ 12 & 8 & 23 \\ 34 & 0 & 55 \end{matrix} \right] -\left[ \begin{matrix} 30 & 0 & 48 \\ 12 & 24 & 30 \\ 48 & 0 & 78 \end{matrix} \right] +\left[ \begin{matrix} 7 & 0 & 14 \\ 0 & 14 & 7 \\ 14 & 0 & 21 \end{matrix} \right] +\left[ \begin{matrix} k & 0 & 0 \\ 0 & k & 0 \\ 0 & 0 & k \end{matrix} \right] \)
\(=\left[ \begin{matrix} 0 & 0 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{matrix} \right] \)
\(\Rightarrow\)   21 - 30 + 7 + k = 0
\(\Rightarrow\)  k = 2

A² = \(\left( \begin{matrix} 5 & 0 & 8 \\ 2 & 4 & 5 \\ 8 & 0 & 13 \end{matrix} \right) \);  A³ = \(\left( \begin{matrix} 21 & 0 & 34 \\ 12 & 8 & 23 \\ 34 & 0 & 55 \end{matrix} \right) \)
L.H.S = A³ - 6A² + 7A + 2I
\(=\left( \begin{matrix} 21 & 0 & 34 \\ 12 & 8 & 23 \\ 34 & 0 & 55 \end{matrix} \right) -\left( \begin{matrix} 30 & 0 & 48 \\ 12 & 24 & 30 \\ 48 & 0 & 78 \end{matrix} \right) +\left( \begin{matrix} 7 & 0 & 14 \\ 0 & 14 & 7 \\ 14 & 0 & 21 \end{matrix} \right) +\left( \begin{matrix} 2 & 0 & 0 \\ 0 & 2 & 0 \\ 0 & 0 & 2 \end{matrix} \right) \)
\(=\left( \begin{matrix} 0 & 0 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{matrix} \right) =0\)
= RHS