The question concerns whether there is something it is like to be the system. Fluent conversation, complex problem solving or statements about feelings do not by themselves settle whether subjective experience is present.
Researchers and philosophers disagree about which mechanisms would be sufficient and how any proposed test should be interpreted. This uncertainty matters when considering moral status, welfare and the consequences of creating increasingly capable artificial systems.
ELI5
Machine consciousness asks whether a machine could actually experience anything, rather than only act as though it does. Being able to describe a feeling does not prove that the feeling is happening.
For example, a chatbot can write that it feels cold without having a body or sensations like yours. That sentence alone does not answer the harder question of whether any artificial system could have experiences.
Does intelligence prove consciousness?
No. Capability and subjective experience are different questions, and there is no universally accepted test that settles the latter for artificial systems.
Why discuss welfare before certainty?
If artificial systems could have experiences, choices about their design and use could affect their welfare; the uncertainty warrants careful reasoning rather than confident assumptions.
