Can conscience command something immoral?

Consider the following claim:
(*) It is never immoral to do what conscience commands.
It would be nice if (*) were true. For instance, it would allow us to defend the duty to obey conscience against the objection that sometimes it is immoral to follow conscience. (Another way to defend the duty to obey conscience is to allow, with Mark Murphy, that sometimes one can both have a duty to do something and a duty to refrain from doing it.) It seems on the face of it that there are only two plausible views on which (*) comes out true. The first is (individual) relativism on which morality is defined by the dictates of one's conscience, which is thus infallible. The second is a version of disjunctivism according to which some beliefs about what one ought to do come from conscience and others come from merely apparent conscience, which are two distinct sources of moral belief (the second may actually be a mess of different sub-sources), with conscience being infallible. Unless the disjunctivism comes along with an infallible criterion for distinguishing the two sources, it's not going to be very useful in practice, since one then won't be able to tell if a given moral belief is a dictate of conscience or only seems to be so.

However, even if one rejects relativism and disjunctivism, (*) is not quite as absurd as it may seem. The main reason to reject (*) is due to counterexamples, but I am actually not able to come up with clear counterexamples against (*) once one takes into account Thomistic/Kantian insights about the importance of maxims to the individuation of action types. The maxim of an action is a description of the action which includes the end, means and reasons for the action; it is something from which it is clear why the action is done. Let's now try two apparent counterexamples to (*):

• Hauptsturmfuehrer Mueller believes he is obliged to kill Jews. But what is Mueller's actual maxim--what is the description under which the actions are seen as obligatory, and which explains why the action is being done by him? Mueller does not wish to kill Jews because the word "Juden" has five letters or because Jews are descendants of Abraham. He wishes to kill Jews because, let's say, he believes Jews are diabolical subhumans. If so, then the action he believes himself to be obliged to do is something like: kill Jews who are diabolical subhumans in order to improve the world. But now that we have attended to the maxim conscience commands Mueller to act under, we see that what he is commanded by conscience to do is impossible, but not actually immoral. If, per impossibile, there were a diabolical subhuman Jew killing whom would improve the world, to kill him would be permissible, indeed laudatory. But it is logically impossible for there to be such (since no subhuman can be Jewish, as only humans can be Jewish). Suppose, then, Mueller moved by his maxim kills a Jewish neighbor. Then, Mueller has indeed done something immoral. But he has not done what his conscience commanded. For his conscience commanded him to kill someone who is a diabolical subhuman, and in killing his neighbor he did no such thing, though he thought he did.
• Dr. Smith believes she should do non-consensual dangerous medical experiments on George for the greater good of humankind. The difference between the case of Smith and that of Mueller is that the hauptsturmfuehrer was acting from the basically correct moral principle that dangerous subhumans should be killed, a principle we affirm when we kill a tiger leaping at us, but was misapplying the principle. The ruthless doctor, however, is a utilitarian, and while her principle that the welfare of one may be sacrificed for the greater good of many is mistaken, her application of it is correct. However, I think a variant of the move made in the preceding case can be made here. Dr. Smith recognizes the fact that George loses out in the transaction. (If she doesn't, the case becomes very similar to that of Mueller.) Now she is acting in conscience. Thus she is not merely acting in a way that harms George because she doesn't care about George, the way an akratic agent might. Rather, she recognizes that she is harming George, but believes that the harm is justified by the greater benefit to humankind. If so, then the maxim is really something like this: do medical experiments on George whose danger to George is morally outweighed by the greater good of humankind. But once again, were the danger indeed morally outweighed by the greater good of humankind, Dr. Smith's actions would be right. But in fact the danger is not outweighed, because serious bodily harms to one person are not morally outweighed in the relevant sense by benefits to another (in the way in which an inconvenience might be morally outweighed by benefits to others; it would be perfectly fine for Dr. Smith to inconvenience George, say by making him wait for an appointment, while trying to find a cure for cancer), and so what Dr. Smith does in fact once again does not fall under her own maxim, even though she thinks she does. What conscience commands her to do is not immoral, though under the circumstances it may be impossible, and what she does is immoral, but is not what conscience commanded her.
  Objection: The maxim does not say that the danger to George is morally outweighed, but simply that it is outweighed in the utilitarian sense, in that the expected disutility to George is less than the expected utility to others.
  Response: Maybe. But if so, then Smith's real maxim will include the truth of utilitarianism--it will be something like: do medical experiments on George whose expected disutility to George is outweighed by the expected utility to others and thereby partially fulfill the duty to maximize total utility. And this maxim, once again, does not command something immoral, but something impossible, since it is impossible to partially fulfill the duty to maximize total utility since there is no such duty. Once again, what Smith does is immoral, but does not accord with her maxim.

It's also worth noting that the objection that this is not how Mueller and Smith explicitly formulate their maxim to themselves is beside the point. For as we can learn from Kant or Freud (or maybe Leibniz or Spinoza as well), we frequently do not know which motives we act on.

Now one might say that the above interpretations are somewhat strained. Maybe. But there is a good argument for thinking something like them is true. The argument goes as follows: if (*) is true, and both relativism and disjunctivism are false, then some such interpretation must be right. One might ask why we should believe (*). But there is very good reason to believe (*), namely the plausibility of the following argument:

1. It is always immoral to refrain from obeying conscience.
2. It is never immoral to refrain from doing something immoral.
3. Therefore, to obey conscience is never something immoral.

That said, I still need to bite the bullet in one respect if I accept this argument. The interpretations of maxims that I gave imply that sometimes conscience requires me to do something impossible, and hence that ought does not imply can. And there I do need to bite the bullet, after having softened it slightly by noting that although one can be obliged to do something impossible, one cannot be culpable for failing.

Let me end with a last objection. Can't we likewise say that it is a duty to try to follow conscience, and that Mueller and Smith are doing something obligatory which they succeed at, namely they really do try to follow conscience? But this generates a problem. For is not their trying to follow conscience the same action as their murder or medical experiments, so that the same action is obligatory and yet wrong? No! The action of trying to follow conscience starts before they start the immoral actions, since trying to follow conscience includes an attempt to discern which of the options before one is in accord with conscience, an attempt that fails.

Note 1: I am not claiming (*) is true, just that it is not as absurd it may seem.

Note 2: I think my suggestion is rather in the spirit of Spinoza's account in the Treatise on the Emendation of the Intellect of how all that we call error is just confusion. I am certainly not claiming that Spinoza is right about that, but that, too, is not quite as absurd as it seems.