scorer module

class spark_matcher.scorer.Scorer(spark_session: pyspark.sql.session.SparkSession, binary_clf: Optional[sklearn.base.BaseEstimator] = None)

Bases: object

fit(X: numpy.ndarray, y: numpy.ndarray) → spark_matcher.scorer.scorer.Scorer

This method fits a clf model on input data X nd the binary targets y.

Parameters

• X – training data

• y – training targets, containing binary values

Returns

The object itself

predict_proba(X: Union[pyspark.sql.column.Column, numpy.ndarray]) → Union[pyspark.sql.column.Column, numpy.ndarray]

This method implements the abstract predict_proba method. It predicts the ‘probabilities’ of the target class for given input data X.

Parameters

X – input data

Returns

the predicted probabilities