U ʹj[L@sZddlmZddlZddlZddlZddlmZddlm Z ddl m Z ddl m Z mZmZmZmZddlmZmZddlmZmZmZmZmZdd lmZdd lmZdd lm Z m!Z!m"Z"dd l#m$Z$e!%e&Z'dZ(erz ddl)Z(Wn$e*k rddl+Z,e,j(Z(YnXe$e$d ddZ-d)ddZ.ddZ/ddZ0d*e e1ee2dddZ3e-d+ee4e feee4e fe1e1eee5e4fdddZ6dd d!d"Z7e"e-dd#ddddddddddddd$e4ee2e4ee1ee4ee4ee4ee1eeee4e4feeee4e4feeee4e4fee4e1eee5e4fe1d%d&d'Z8Gd(ddeZ9dS),Nwraps)Path)copytree)AnyDictListOptionalUnion) ModelHubMixinsnapshot_download)get_tf_versionis_graphviz_availableis_pydot_availableis_tf_available yaml_dump) constants)HfApi)SoftTemporaryDirectoryloggingvalidate_hf_hub_args) CallableT)fnreturncstfdd}|S)Ncs,t|dstdjd|f||S)Nhistoryz Cannot use 'zp': Keras 3.x is not supported. Please save models manually and upload them using `upload_folder` or `hf upload`.)hasattrNotImplementedError__name__)modelargskwargsrZ/var/www/html/bwave/python/venv/lib/python3.8/site-packages/huggingface_hub/keras_mixin.py_inner+s   z'_requires_keras_2_model.._innerr)rr%r#r"r$_requires_keras_2_model)sr&cCsdg}|D]N\}}|r&|d|n|}t|tjrL|t||q |||fq t|S)aFlatten a nested dictionary. Reference: https://stackoverflow.com/a/6027615/10319735 Args: dictionary (`dict`): The nested dictionary to be flattened. parent_key (`str`): The parent key to be prefixed to the children keys. Necessary for recursing over the nested dictionary. Returns: The flattened dictionary. .)items isinstance collectionsMutableMappingextend _flatten_dictappenddict) dictionaryZ parent_keyr)keyvalueZnew_keyr#r#r$r.7s r.cCs`d}|jdk r\|j}t|}tjj|d<d}|D]\}}|d|d|d7}q<|S)z6Parse hyperparameter dictionary into a markdown table.NZtraining_precisionz*| Hyperparameters | Value | | :-- | :-- | z| z | z | )Z optimizerZ get_configr.kerasZmixed_precisionZ global_policynamer))rtableZoptimizer_paramsr2r3r#r#r$_create_hyperparameter_tableTs  r7c Cs(tjj||ddddddddd dS)N /model.pngFTTB`)Zto_fileZ show_shapesZ show_dtypeZshow_layer_namesZrankdirZ expand_nestedZdpiZ layer_range)r4utils plot_model)rsave_directoryr#r#r$ _plot_networkbsr>T)repo_dirr<metadatacCs|d}|rdSt|}|r6tr6tr6t|||dkrBi}d|d<d}|t|dd7}|d7}|d7}|d 7}|d 7}|dk r|d 7}|d 7}|d 7}||7}|d7}|rtj|dr|d7}|d7}|d7}d}|d|d7}|d7}||dS)zd Creates a model card for the repository. Do not overwrite an existing README.md file. z README.mdNr4 library_namez--- F)Zdefault_flow_stylez/ ## Model description More information needed z9 ## Intended uses & limitations More information needed z: ## Training and evaluation data More information needed z ## Training procedure z ### Training hyperparameters z; The following hyperparameters were used during training:  r8z ## Model Plot z
z$ View Model Plot z ./model.pngz ![Model Image](z) z
) existsr7rrr>rospath write_text)rr?r<r@Z readme_pathZhyperparametersZ model_cardZ path_to_plotr#r#r$_create_model_cardps:  rGF)r=configinclude_optimizerr<tagsc Kstdkrtd|jstdt|}|jddd|r~t|tsVtdt |d|t j  d}t ||W5QRXi}t|tr||d <nt|tr|g|d <|d d} | dk rtd td |kr|d | n | g|d <|jdk rT|jjikrT|d } | r$td t| j ddd}t j|jj|dddW5QRXt||||tjj||fd|i|dS)aL Saves a Keras model to save_directory in SavedModel format. Use this if you're using the Functional or Sequential APIs. Args: model (`Keras.Model`): The [Keras model](https://www.tensorflow.org/api_docs/python/tf/keras/Model) you'd like to save. The model must be compiled and built. save_directory (`str` or `Path`): Specify directory in which you want to save the Keras model. config (`dict`, *optional*): Configuration object to be saved alongside the model weights. include_optimizer(`bool`, *optional*, defaults to `False`): Whether or not to include optimizer in serialization. plot_model (`bool`, *optional*, defaults to `True`): Setting this to `True` will plot the model and put it in the model card. Requires graphviz and pydot to be installed. tags (Union[`str`,`list`], *optional*): List of tags that are related to model or string of a single tag. See example tags [here](https://github.com/huggingface/hub-docs/blob/main/modelcard.md?plain=1). model_save_kwargs(`dict`, *optional*): model_save_kwargs will be passed to [`tf.keras.models.save_model()`](https://www.tensorflow.org/api_docs/python/tf/keras/models/save_model). Nz>Called a Tensorflow-specific function but could not import it.z+Model should be built before trying to saveT)parentsexist_okzAProvided config to save_pretrained_keras should be a dict. Got: ''wrJ task_namez>`task_name` input argument is deprecated. Pass `tags` instead.z history.jsonzZ`history.json` file already exists, it will be overwritten by the history of this version.zutf-8)encoding)indent sort_keysrI)r4 ImportErrorZbuilt ValueErrorrmkdirr*r0 RuntimeErrortyperZ CONFIG_NAMEopenjsondumpliststrpopwarningswarn FutureWarningr/rrC UserWarningrGmodelsZ save_model) rr=rHrIr<rJmodel_save_kwargsfr@rOrEr#r#r$save_pretrained_kerassJ#          rfKerasModelHubMixin)rcOs tj||S)a Instantiate a pretrained Keras model from a pre-trained model from the Hub. The model is expected to be in `SavedModel` format. Args: pretrained_model_name_or_path (`str` or `os.PathLike`): Can be either: - A string, the `model id` of a pretrained model hosted inside a model repo on huggingface.co. Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a user or organization name, like `dbmdz/bert-base-german-cased`. - You can add `revision` by appending `@` at the end of model_id simply like this: `dbmdz/bert-base-german-cased@main` Revision is the specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any identifier allowed by git. - A path to a `directory` containing model weights saved using [`~transformers.PreTrainedModel.save_pretrained`], e.g., `./my_model_directory/`. - `None` if you are both providing the configuration and state dictionary (resp. with keyword arguments `config` and `state_dict`). force_download (`bool`, *optional*, defaults to `False`): Whether to force the (re-)download of the model weights and configuration files, overriding the cached versions if they exist. proxies (`Dict[str, str]`, *optional*): A dictionary of proxy servers to use by protocol or endpoint, e.g., `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request. token (`str` or `bool`, *optional*): The token to use as HTTP bearer authorization for remote files. If `True`, will use the token generated when running `transformers-cli login` (stored in `~/.huggingface`). cache_dir (`Union[str, os.PathLike]`, *optional*): Path to a directory in which a downloaded pretrained model configuration should be cached if the standard cache should not be used. local_files_only(`bool`, *optional*, defaults to `False`): Whether to only look at local files (i.e., do not try to download the model). model_kwargs (`Dict`, *optional*): model_kwargs will be passed to the model during initialization > [!TIP] > Passing `token=True` is required when you want to use a private > model. )rgZfrom_pretrained)r r!r#r#r$from_pretrained_kerass2rhz'Push Keras model using huggingface_hub.)rHcommit_messageprivate api_endpointtokenbranch create_prallow_patternsignore_patternsdelete_patternslog_dirrIrJr<)repo_idrHrirjrkrlrmrnrorprqrrrIrJr<c Kst|d}|j|||ddj}t}t||}t||f|| ||d|| dk r| dkrbgnt| trr| gn| } | dt | |d|j d||||||| | | d W5QRSQRXdS) a Upload model checkpoint to the Hub. Use `allow_patterns` and `ignore_patterns` to precisely filter which files should be pushed to the hub. Use `delete_patterns` to delete existing remote files in the same commit. See [`upload_folder`] reference for more details. Args: model (`Keras.Model`): The [Keras model](`https://www.tensorflow.org/api_docs/python/tf/keras/Model`) you'd like to push to the Hub. The model must be compiled and built. repo_id (`str`): ID of the repository to push to (example: `"username/my-model"`). commit_message (`str`, *optional*, defaults to "Add Keras model"): Message to commit while pushing. private (`bool`, *optional*): Whether the repository created should be private. If `None` (default), the repo will be public unless the organization's default is private. api_endpoint (`str`, *optional*): The API endpoint to use when pushing the model to the hub. token (`str`, *optional*): The token to use as HTTP bearer authorization for remote files. If not set, will use the token set when logging in with `hf auth login` (stored in `~/.huggingface`). branch (`str`, *optional*): The git branch on which to push the model. This defaults to the default branch as specified in your repository, which defaults to `"main"`. create_pr (`boolean`, *optional*): Whether or not to create a Pull Request from `branch` with that commit. Defaults to `False`. config (`dict`, *optional*): Configuration object to be saved alongside the model weights. allow_patterns (`List[str]` or `str`, *optional*): If provided, only files matching at least one pattern are pushed. ignore_patterns (`List[str]` or `str`, *optional*): If provided, files matching any of the patterns are not pushed. delete_patterns (`List[str]` or `str`, *optional*): If provided, remote files matching any of the patterns will be deleted from the repo. log_dir (`str`, *optional*): TensorBoard logging directory to be pushed. The Hub automatically hosts and displays a TensorBoard instance if log files are included in the repository. include_optimizer (`bool`, *optional*, defaults to `False`): Whether or not to include optimizer during serialization. tags (Union[`list`, `str`], *optional*): List of tags that are related to model or string of a single tag. See example tags [here](https://github.com/huggingface/hub-docs/blob/main/modelcard.md?plain=1). plot_model (`bool`, *optional*, defaults to `True`): Setting this to `True` will plot the model and put it in the model card. Requires graphviz and pydot to be installed. model_save_kwargs(`dict`, *optional*): model_save_kwargs will be passed to [`tf.keras.models.save_model()`](https://www.tensorflow.org/api_docs/python/tf/keras/models/save_model). Returns: The url of the commit of your model in the given repository. )ZendpointT)rsrlrjrL)rHrIrJr<Nzlogs/*Zlogsr) Z repo_typersZ folder_pathrirlrevisionrnrorprq) rZ create_reporsrrrfr*r]r/rZ upload_folder)rrsrHrirjrkrlrmrnrorprqrrrIrJr<rdapitmpZ saved_pathr#r#r$push_to_hub_keras%sHP    rwc@s8eZdZdZddZedeeee fdddZ dS) rgaA Implementation of [`ModelHubMixin`] to provide model Hub upload/download capabilities to Keras models. ```python >>> import tensorflow as tf >>> from huggingface_hub import KerasModelHubMixin >>> class MyModel(tf.keras.Model, KerasModelHubMixin): ... def __init__(self, **kwargs): ... super().__init__() ... self.config = kwargs.pop("config", None) ... self.dummy_inputs = ... ... self.layer = ... ... def call(self, *args): ... return ... >>> # Initialize and compile the model as you normally would >>> model = MyModel() >>> model.compile(...) >>> # Build the graph by training it or passing dummy inputs >>> _ = model(model.dummy_inputs) >>> # Save model weights to local directory >>> model.save_pretrained("my-awesome-model") >>> # Push model weights to the Hub >>> model.push_to_hub("my-awesome-model") >>> # Download and initialize weights from the Hub >>> model = MyModel.from_pretrained("username/super-cool-model") ``` cCst||dS)N)rf)selfr=r#r#r$_save_pretrainedsz#KerasModelHubMixin._save_pretrainedN)rHc KsLtdkrtdtj|s2t|||dtd} n|} tj| } | | _ | S)aHere we just call [`from_pretrained_keras`] function so both the mixin and functional APIs stay in sync. TODO - Some args above aren't used since we are calling snapshot_download instead of hf_hub_download. Nz>Called a TensorFlow-specific function but could not import it.r4)rsrt cache_dirrAZlibrary_version) r4rTrDrEisdirr r rcZ load_modelrH) clsZmodel_idrtrzZforce_downloadproxiesZresume_downloadZlocal_files_onlyrlrHZ model_kwargsZstorage_folderrr#r#r$_from_pretraineds  z#KerasModelHubMixin._from_pretrained)N) r __module__ __qualname____doc__ry classmethodr rr]rr~r#r#r#r$rgs#  )r')TN)NFTN):collections.abcabcr+rZrDr_ functoolsrpathlibrshutilrtypingrrrr r Zhuggingface_hubr r Zhuggingface_hub.utilsr rrrrr'rZhf_apirr;rrrZ utils._typingrZ get_loggerrloggerr4Ztf_kerasrTZ tensorflowtfr&r.r7r>boolr0rGr]r\rfrhrwrgr#r#r#r$s            , S5z